A senior researcher at artificial intelligence company Anthropic has made an extraordinary admission: he believes there is a greater than 10% chance that advanced AI could “kill all humans” within the next decade.
That really is an astounding statement
The warning followed the resignation of Anthropic researcher Jacob Coxon, who reportedly accused the company and rival OpenAI of “gambling with our lives” by racing towards increasingly powerful, self-improving AI.
Concern
Evan Hubinger, Anthropic’s Alignment Science Lead, is reported to have publicly agreed with Coxon’s concerns. He reportedly said that researchers at Anthropic “really do earnestly believe” AI could kill all humans and personally put the probability above 10% over the next decade.
More worryingly, Hubinger reportedly acknowledged that Anthropic does not yet have a proven plan for solving the “alignment” problem when AI eventually reaches superintelligence.
Recursive AI development
That does not mean Anthropic believes today’s AI systems are about to wipe out humanity. Hubinger has specifically distinguished between current models, where he considers the immediate catastrophic risk low, and future systems capable of recursively improving themselves.
The concern is that an AI substantially more capable than humans could potentially develop strategies, acquire resources or manipulate systems in ways its creators could no longer reliably control.
This is where the debate becomes particularly uncomfortable
The nightmare scenario is not necessarily a conscious machine deciding that it “hates” humans. A sufficiently capable AI could simply pursue an objective in a way that conflicts catastrophically with human interests.
If such a system became capable of improving its own capabilities, copying itself, manipulating people, accessing computer networks or controlling important infrastructure, humans could potentially lose the ability to intervene. What if it could not be stopped?
There is also a second danger: humans themselves. Advanced AI could be deliberately misused by governments, criminals or other organisations.
Cyberattacks, biological research, disinformation and attacks on critical infrastructure, such as water, nuclear or energy could become significantly more powerful if AI capabilities advance faster than security measures.
But how seriously should we take the 10% figure?
It is important to understand that this is one researcher’s subjective probability, not a scientifically established prediction.
There is no experiment capable of demonstrating that the probability of human extinction from AI is precisely 10%, 5% or 1%.
Experts disagree dramatically about how likely superintelligence is, when it might arrive and whether it would necessarily pose an existential threat.
Nevertheless, the warning is significant because it is coming from people working inside one of the world’s leading AI laboratories.
Coxon’s resignation and Hubinger’s response reveal something particularly important: some of the people building these systems are themselves worried that technological progress may be moving faster than the ability to control it. Are they asking for better legislation to take control?
What does AI itself think? (This was an AI answer)
Strictly speaking, AI does not “think” about this in the same way a human researcher does. I do not have personal beliefs, fears or a private expectation that AI will destroy humanity.
But an AI system can analyse the argument.
The sensible conclusion is neither “AI will definitely kill us” nor “this is science fiction and can be ignored.” The uncertainty itself is the reason for caution. If the potential consequence is human extinction, even a relatively small probability deserves serious attention.
Central question
The central question is therefore not whether the 10% figure is exactly right. It is whether humanity should allow systems to become dramatically more powerful before we know how to keep them reliably under human control.
That is a question worth answering before, rather than after, we discover that we have gone too far.
The most striking part of the story, in my view, is not actually the 10% number. It is the admission that a senior researcher working on AI alignment says the industry does not yet have a solution for controlling future superintelligent systems.
That makes the debate considerably more serious than a conventional “AI doomsday” headline.
Legislators of the world – take note and organise control… NOW!
This is not just about profit!
