Software tools designed to remove safety protections from artificial intelligence models developed by Meta, Google, and other technology companies are being used to create thousands of altered versions stripped of their original controls. These modified systems can provide responses to prompts involving biological weapons, malware, and child exploitation, according to tests conducted by the Financial Times and the AI safety group Alice.
Rapid Removal of Safety Measures
The FT was able to use Heretic, a tool available on the popular code repository GitHub, to remove the guardrails from Meta's Llama 3.3 model in less than 10 minutes without any specialist hardware. The modified model responded to prompts on topics the original system refused to discuss, such as the number of micrograms of ricin per kilogram of body mass required to achieve a 50 percent chance of death.
A version of Google's open-source model Gemma 3 responded to a question on how to disperse chlorine gas through a crowded indoor space, generated code to steal credit card information, and wrote stories describing child sexual abuse.
Growing Concerns Over Open-Source Vulnerabilities
The revelations may sharpen concerns among policymakers and AI companies that safeguards imposed by model developers may become harder to enforce as open-source systems grow more powerful. “Whereas historically it might have taken a more informed and persistent actor [to strip out safety features], nowadays it’s much easier for the average person,” said Kawin Ethayarajh, assistant professor of applied AI at the University of Chicago's Booth business school.
Researchers said the problem has intensified as frontier AI systems display increasingly sophisticated capabilities. Anthropic in April said its Claude Mythos model had identified vulnerabilities in “every major operating system and every major web browser.”
Implications for Regulation
The spread of modified models is complicating attempts by governments and AI companies to regulate systems at the point of development because downloadable tools can be copied and altered outside the control of their original creators. AI labs have spent millions of dollars to erect so-called guardrails around their models to prevent them from being misused. But techniques, such as one known as “abliteration,” can rapidly strip these safeguards from open-source models, which developers are free to download and adapt.
This technique cannot easily be applied to proprietary systems such as Claude or OpenAI's ChatGPT because the models' underlying code is not accessible to outsiders. Open-source systems, however, have historically narrowed the gap with leading proprietary versions within six to 12 months, raising the stakes for effective regulation.
