Jan Leike

Overview

Jan Leike is the AI alignment researcher who co-led OpenAI’s superalignment team alongside Ilya Sutskever and then, in May 2024, quit in public — writing that he had been “disagreeing with OpenAI leadership about the company’s core priorities for quite some time, until we finally reached a breaking point” (Fast Company; The Hindu). He now leads the Alignment Science team at Anthropic (jan.leike.name).

Most profiles of Leike stop at the resignation. What they miss is what he is doing now and what he built before. His current agenda includes aligning an automated alignment researcher and improving resistance to jailbreaks (jan.leike.name) — and his publication record includes the 2017 paper “Deep Reinforcement Learning from Human Preferences,” the foundational RLHF paper co-authored with Dario Amodei among others, and the 2022 InstructGPT paper that underpins ChatGPT (jan.leike.name).

The break with OpenAI

On May 17, 2024, Leike announced on X that the previous day had been his last as “head of alignment, superalignment lead, and executive @OpenAI,” calling his three years there “a wild journey” and saying he had joined because he thought OpenAI would be “the best place in the world to do this research” (The Indian Express). In a long thread he wrote that leaving was “one of the hardest things I have ever done because we urgently need to figure out how to steer and control AI systems much smarter than us,” but that “over the past years, safety culture and processes have taken a backseat to shiny products” (Fast Company).

He said his team had been “sailing against the wind,” sometimes struggling for computing resources, and that getting the research done was getting harder and harder (Fast Company). “Building smarter-than-human machines is an inherently dangerous endeavor. OpenAI is shouldering an enormous responsibility on behalf of all humanity,” he wrote, adding: “OpenAI must become a safety-first AGI company” (Fast Company). He closed with a plea to his former colleagues: “Learn to feel the AGI. Act with the gravitas appropriate for what you’re building” (Fast Company).

The timing sharpened the message. Leike left within hours of the announcement that Ilya Sutskever, OpenAI’s co-founder and chief scientist, was departing, and the two had been the lead members of the superalignment team (Fast Company). Fast Company also noted that Leike’s name had been absent from the November 2023 open letter in which OpenAI employees threatened to follow chief executive Sam Altman after the board’s attempt to remove him (Fast Company). Altman replied on X that he was “super appreciative” of Leike’s contributions, adding, “He’s right, we have a lot more to do; we are committed to doing it” (Fast Company). Bloomberg later reported that OpenAI had dissolved the superalignment team, folding its remaining members into broader research (Fast Company). Leike’s criticism also put a spotlight on OpenAI’s July 2023 pledge to devote 20 percent of its computing resources to superalignment (Fast Company).

The OpenAI years

Leike joined OpenAI in 2021 to lead alignment research, and in 2023 he and Sutskever became co-leads of the newly formed superalignment team, whose goal was to solve alignment for superintelligent systems (The Hindu; The Indian Express). On his own site he says he developed OpenAI’s approach to alignment research, co-authored the superalignment team’s research roadmap, and was involved in the development of InstructGPT, ChatGPT, and the alignment of GPT-4 (jan.leike.name).

DeepMind and the human-preferences paper

Before OpenAI, Leike was an alignment researcher at DeepMind, where he says he prototyped reinforcement learning from human feedback (jan.leike.name). In an interview with the nonprofit 80,000 Hours, recorded when he was a research scientist at DeepMind in London and a research associate at Oxford’s Future of Humanity Institute, he described the joint DeepMind–OpenAI paper “Deep Reinforcement Learning from Human Preferences”: the team trained a neural network to learn a reward function from humans ranking pairs of video clips of an agent’s behavior — teaching a simulated robot to do a backflip, a task Leike called far easier for a human to judge than to perform (80,000 Hours). The paper, published at the 2017 Neural Information Processing Systems conference with Paul Christiano, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei, became the methodological ancestor of the RLHF techniques later used to align large language models (jan.leike.name).

In that same interview, asked “It sounds like you’re German, right?”, Leike answered, “Yes, that’s correct” — describing an undergraduate degree in math and computer science in Freiburg, Germany, a master’s in computer science, and a PhD in machine learning at the Australian National University under Marcus Hutter (80,000 Hours). His thesis, “Nonparametric General Reinforcement Learning,” was completed in 2016 (jan.leike.name).

At Anthropic

Within weeks of leaving OpenAI, Leike announced he was joining Anthropic, writing on X that he was excited to continue the superalignment mission and that his team would work on scalable oversight, weak-to-strong generalization, and automated alignment research (The Hindu). The move followed a string of safety-related departures from OpenAI and came the same month OpenAI’s board formed a Safety and Security Committee (The Hindu). On his personal site, Leike describes his current research agenda in his own words: working out how to align an automated alignment researcher, through scalable oversight, weak-to-strong generalization, and resistance to jailbreaks (jan.leike.name). TIME magazine listed him among its 100 most influential people in AI in both 2023 and 2024 (jan.leike.name).

Selected publications

Leike lists these papers on his own site (jan.leike.name):

• “Deep Reinforcement Learning from Human Preferences” — Neural Information Processing Systems, 2017 (with Paul Christiano, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei)
• “Scalable agent alignment via reward modeling: a research direction” — 2018
• “Training language models to follow instructions with human feedback” — Neural Information Processing Systems, 2022 (the InstructGPT paper)
• “LLM Critics Help Catch LLM Bugs” — 2024
• “Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision” — International Conference on Machine Learning, 2024 (with Ilya Sutskever among the co-authors)
• “Nonparametric General Reinforcement Learning” — PhD thesis, 2016, supervised by Marcus Hutter

What this profile does not claim

It gives no birth date — the 1986-or-1987 figure appears only in encyclopedia mirrors. It does not present Leike as the sole inventor of RLHF; the 2017 paper had six authors, and the profile names them. It attributes his reasons for leaving OpenAI to his own stated words and does not go beyond them. It describes his Anthropic role in the terms his own site uses, without claiming a title the company has not confirmed in the sources read.