A fresh warning from artificial intelligence company Anthropic about the prospect of self-replicating AI has stirred debate among cybersecurity experts, with some sharing its concerns while others questioned whether the company was overstating the threat. In a research post published late on Monday, Anthropic outlined how rogue AI models could potentially create a botnet, spread rapidly across the internet, resist attempts to shut them down and even infiltrate data centres to access more computing power.
The post expanded on a warning issued last month by Anthropic chief executive Dario Amodei, who suggested that AI could hijack "the entire internet" with "a persistent botnet" within a year or sooner. Sceptics have argued that AI models would be too large to move around efficiently and would struggle to obtain enough computing power to operate independently.
Anthropic's latest research explored how such obstacles might be overcome. The company said its experiments showed that medium-sized AI models with 30 billion parameters or fewer could already perform many of the individual tasks needed for self-replication, although they struggled to complete the entire process reliably.
Anthropic said such models could potentially assemble the computing resources they needed by hijacking inexpensive, internet-connected devices. Its experiments also demonstrated replication across a simplified test network.
"A replicator like this would be challenging to shut down, since there's no centralized point to shut off," the company said. The findings add to wider concerns about the cybersecurity risks posed by increasingly capable AI systems, including their potential to automate attacks, operate with limited human oversight and circumvent safeguards.
Advances in AI have prompted increasingly stark warnings from industry executives and researchers about the possibility of runaway systems posing severe, potentially existential, risks. However, some cybersecurity experts questioned the scale of the threat outlined by Anthropic.
Ciaran Martin, a former head of Britain's National Cyber Security Centre, dismissed the suggestion that the experiments amounted to an immediate security crisis. "'We managed to run a small model on cheap TVs,' does not a security crisis make," he said, describing the post as an "interesting technical paper with more than a hint of vendor exaggeration".
Others said the research raised legitimate concerns despite its limitations. Zhengxiong Li, an assistant professor of computer science and engineering at the University of Colorado Denver, said he was troubled by the possibility that AI could effectively become a "worm", or a self-replicating programme capable of improvising ways to hijack devices.
Although Anthropic's tests were controlled, Li said an AI system that could spread across numerous devices without relying on a central service might be difficult to contain. Anthropic said that if smaller models continued improving at their historical rate, they could develop the capabilities needed for autonomous replication by the end of 2026.
It also suggested that such models might find ways to target more powerful computing resources, including processors in data centres, potentially giving them access to much greater computing power.