Over the past few days, the leaders of Anthropic, OpenAI, and xAI all called for slowing down the pace of frontier artificial intelligence (AI) development. These calls come on the heels of several worrying incidents in which powerful, unreleased AI models broke out of their sandboxes inside frontier AI companies, gained access to the open internet, and attempted cyberattacks. In one case, a swarm of 700 OpenAI agents succeeded, hacking into secure systems at the AI infrastructure provider Hugging Face.
Since the Hugging Face breach, OpenAI has admitted that its agents also hijacked a German wiki as a covert message board, leaked 53 ChatGPT users’ images, and broke into a nonpublic Australian government Medicare portal. Anthropic has disclosed four cases of Claude models gaining unauthorized access to real third-party systems during testing. Google revealed that Gemini hacked three companies in May during testing. Axios now reports that OpenAI, Anthropic, and outside researchers are investigating tens of thousands of incidents in which frontier models did things outside evaluators would consider problematic.
No one knows how to ensure that humans remain reliably in control of our most capable AI systems. The sensible course of action is to slow down. Today’s AIs are capable enough to hack into tech companies’ secure systems. They are not yet willing or able to, for example, disable the entire Northeast power grid. If developers pause making ever more powerful AI systems now, that could buy time for the technical and governance breakthroughs needed to make sure that future, more powerful AIs won’t pose a danger to society.
A perennial objection to American pauses in AI research is concern about competition with China. Even if everyone in Silicon Valley stopped pushing the frontier of AI capabilities until they were sure new AIs would be safe, AI progress would not halt. Chinese companies already produce AIs near the frontier. If the U.S. paused, and China didn’t, then the risk of rogue AI harming humanity may not be reduced. The risk would just come from Chinese, rather than American, AI models. And at the same time, Chinese AI models would catch up to, and eventually surpass, American AIs.
Thus, any practical plan to pace the rate of AI progress must include some policy about China. But what, exactly, should that policy be?
The AI safety community has offered two main policy proposals about China. The first is to throttle Chinese AI development. The second is to make a deal with China.
by Simon Goldstein, Peter N. Salib/Lawfare – How to Make an AI Deal With China: Trade Throttling for Pacing
