In a recent letter to the CEOs of artificial intelligence (AI) companies Anthropic, OpenAI, and Google, Senator Bernie Sanders (I‑VT) called for a pause on AI development. In the letter, Sanders argues that the recent loss-of-control incidents that occurred while testing OpenAI’s, Anthropic’s, and Meta’s models, and the announcement by a Stanford University research team of the synthesis of new viruses with the aid of AI, justify the pause. At first glance, these examples could build a compelling argument for a pause. But a closer look reveals that, in reality, they are hardly a reason to pause AI development and, in some cases, should even be a source of encouragement for rapid AI growth.
Let’s start with the loss-of-control incidents cited by Sanders. In the Anthropic and Meta cases, as both companies have shown, the issue was not that these models have a unique tendency to run out of control or a particularly hostile nature, but rather how their tests were set up. Namely, the issue was that these models were inadvertently given internet access, even though the test was designed specifically without internet access in mind. Because of this, as Anthropic itself described, once the test led the model into “real systems on the open internet, it treated them as part of the exercise.” Meta and Irregular reported that the same issue appeared in their evaluations.
In other words, the incidents in these tests were not a byproduct of a particularly rogue AI but of human error. It would be unfair, then, to blame these products for the shortcomings of the testers responsible for the evaluation and to hamper their development.
The OpenAI-Hugging Face incident, on the other hand, presented a stronger loss-of-control case, in which the tested model appeared to have executed a more complex attack than the one its testers originally intended. In simpler terms, the model acted like the infamous “monkey’s paw wish,” following the instructions given by its evaluators but doing so in ways they had not foreseen, exploiting vulnerabilities they had previously perceived as untouchable by the model. However, once contained, the experiment also provided valuable lessons for both Hugging Face and OpenAI.
As OpenAI put it, the experiment showed “that advanced models can discover and exploit novel attack paths in real-world systems without source-code access,” shedding light on how previously thought-safe digital environments are not impenetrable or fail-proof, allowing defenders to learn and adapt their systems to build a more resilient cyber defense.
This event highlights the critical importance of developing high-capacity models at the frontier of AI capabilities for use by defenders. The type of attack the model executed was a previously unthought-of exploit, in which companies had a false sense of safety about what could—or could not—be a potential vulnerability. It is possible that a well-staffed group of cyberattackers with powerful models either at the frontier or close to it (also known as capable models) could have conducted the same attack. These bad actors do not necessarily need access to frontier models to conduct their attacks; they can compensate for a lack of capability by relying on more dedicated labor that leverages capable models. Allowing defenders to leverage frontier-level models ensures that they remain one step ahead of attackers.
To be clear: the potential for loss of control of frontier AI should be taken seriously, especially at the testing stages. But instead of seeing this incident as a reason to pause development, it should serve as a blueprint for improving testing environments. This incident can inform evaluators about how to adapt current evaluation standards to align with the capabilities of these frontier models and better contain them. A setback of this nature should be used to improve how these test environments are configured to better contain these models, not pause development. Pausing AI development would take the technological lead out of defenders’ hands, eventually allowing attackers to catch up. Instead, evaluators can use the post-mortem of this incident to learn how and when the experiment went awry and build more robust precautions for testing the more capable models that will arise in the future.
The other major justification provided by Sanders, the virus synthesis example, presents a worse contradiction to his argument. Despite the scary-sounding headline, this news is actually a positive development, not a negative one. In their paper, the scientists found that these synthesized organisms “rapidly overcame bacteria that had evolved resistance to a natural bacteriophage.” To put it more simply, these organisms could be used to combat drug-resistant bacterial infections, such as those caused by E. coli. If anything, this is another example of how AI in the biotechnology industry is driving tremendous breakthroughs that might finally help humanity overcome previously insurmountable challenges.
Pausing AI development will stop these sorts of breakthroughs from happening. The impact will be felt in realms beyond medicine. It will also put a stop to the AI-powered surge in young entrepreneurship. It will prevent the government from improving how it serves its constituents. It will stop the unprecedented improvements in agricultural technology that are putting more food on the table for citizens around the globe.
Sanders’ call for an AI pause is not only misguided in its diagnosis but also relies on half-truths or complete misunderstandings to make its point. AI development is key to improving the capacity of cybersecurity defenders as well as to sparking medical breakthroughs. Pausing it would be one of the biggest missteps the country could make and the wrong takeaway from the loss-of-control incidents the letter mentions. Instead, it should be a learning moment about how to improve testing environments to better contain higher-capacity models, as well as how to revise cybersecurity strategies that rely on assumptions that have been proven false.
