Anthropic PBC has called for the creation of an industrywide mechanism to pause the development of artificial intelligence, giving society the opportunity to “deal with its immense implications.” The company, known for its popular assistant Claude and the newly released Mythos model, outlined its vision in a lengthy blog post on Thursday.
AI Advancements and Risks
The technology is advancing to the point where AI will make human labour thousands of times more efficient or even replace it, creating a new set of risks, the company said. AI could begin to improve itself and build its own successors, diminishing the role of humans. “That collision, where recursive intelligence building itself ever faster meets the world of humans, relationships, and governance, is another part of this future we can’t predict,” Anthropic wrote.
Comparison to Nuclear Arms Control
The blog post compared the idea to the international regulation of nuclear weapons. Any pause mechanism would also need to ensure less scrupulous AI labs don’t secretly keep working to advance the technology during a break, likely through checks by peer labs. Anthropic said it would organize conversations with policymakers, researchers and other AI companies to discuss a structure for “coordination and deliberation” and will publish the results.
Historical Context and Criticism
Anthropic’s post isn’t the first time a frontier AI researcher has called for a break in development. In 2023, the Future of Life Institute called for a pause of at least six months to put guardrails on AI, warning about potentially catastrophic effects. Billionaire Elon Musk signed it along with more than 1,000 other researchers and executives. Critics at the time said that stopping development would unnecessarily crimp innovation, and that U.S. companies agreeing to tap the brakes would hand an advantage to other countries. Former Google chief executive Eric Schmidt warned that the pause would give an advantage to competitors in China, which is locked in a technological trade war with the U.S.
Challenges of AI Arms Control
In the blog, Anthropic acknowledged this difficulty, and said “arms control” for AI is more challenging than other technologies. “Training runs are far easier to conceal than missile silos, their inputs are general-purpose, and the incentive to defect quietly is enormous, because whoever continues while others pause could inherit the lead,” Anthropic wrote. While the world has built similar regimes in the past, they “took decades to build both the infrastructure and the trust. We don’t have that long.”
Anthropic itself has continued to put out advanced models and tools such as its popular assistant Claude, and the new Mythos model that it says can detect and exploit cybersecurity flaws at remarkable speed. The company is also preparing for an initial public offering.
