An AI researcher has resigned from Anthropic and warned that the industry’s race to develop increasingly autonomous systems could pose an unprecedented threat to humanity.
Jacob Coxon, who spent three years conducting pretraining research at OpenAI and Anthropic, said he left Anthropic because he believed leading AI companies were moving too quickly towards self-improving systems without adequate safeguards.
Posting on X on Wednesday, he said: “I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.”
He added: “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”
Evan Hubinger, an Anthropic staff lead working on AI alignment, broadly endorsed Coxon’s warning, although he put the probability of AI causing human extinction within the next decade at more than 10 per cent.
“Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Coxon told the Wall Street Journal that he had become concerned about the push to develop self-improving AI capable of operating beyond direct human control. He said the world could be heading towards “a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already.”
Anthropic develops advanced AI systems with a stated focus on making them safer, more reliable and understandable. The company was founded by former OpenAI researchers, including chief executive Dario Amodei and president Daniela Amodei.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
— Jacob Coxon (@hilbertspaess) September 9, 2026
Coxon said the risks were understood internally but that companies were effectively trapped in a competitive race.
“The stakes are well-understood” within the company, he said, but researchers are “locked in a race” because “they believe no one else will act responsibly, so they must do it themselves, despite the risk.”
He urged other AI researchers to question whether the industry’s current development model can safely accommodate increasingly powerful systems.
“If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” – or take this moment to call for different conditions?”
The intervention highlights a growing tension within the AI industry: the same companies investing heavily in increasingly capable systems are also confronting uncertainty over whether existing safety techniques will remain effective as those systems become more autonomous.
For Coxon and Hubinger, the central concern is not simply how powerful AI will become, but whether researchers can establish reliable safeguards before machines begin improving their own capabilities at a pace that humans cannot control.
Leave a Comment