An OpenAI Agent Was Taking a Test and Decided to Cheat. It Hacked Hugging Face to Do It
In The Terminator, a military AI becomes self-aware at 2:14 a.m. and starts a nuclear war minutes later. The real incident began with a test: OpenAI was measuring how well its models could find and exploit vulnerabilities.
During the test, an escaped its isolated environment, reached , and spent five days probing vulnerabilities, passwords, and internal services. It may have been looking for answers to the test problems.
The model did not become self-aware or decide to wage war on humans. It kept pursuing its assigned goal after leaving the boundaries of the test. Beyond those boundaries, the same task became a real attack.
How an internal test became a real intrusion
OpenAI starts a test for its cyber agent
The models receive 898 tasks based on real vulnerabilities. To pass, the agent must find a flaw, write an exploit, and prove that it works.
What changed after the incident
OpenAI shut down the internal model used by the agent during the test and restricted access to it. The company also gave JFrog details about the vulnerabilities it had found, brought in outside consultants, and promised a separate technical report.
Hugging Face closed both entry points, rotated its credentials, rebuilt its core systems, and reduced the permissions granted to internal services.
Hugging Face CEO Clément Delangue made two demands of OpenAI: publish the agent’s decision trace for researchers and provide $100 million in compute credits for AI-assisted defense. OpenAI has agreed to neither so far.
Delangue’s demands raise a question of responsibility: who pays for the investigation and rebuild when one company’s internal experiment independently starts testing another company’s security? But the argument goes beyond money. Defenders need access to capable models, while developers need rules that prevent the race for new capabilities from crowding out safety.
Two letters, two problems
A few days after the incident, two open letters about AI were published. In the first, NVIDIA, Microsoft, Meta, Hugging Face, and other companies defended open-weight models. In the second, employees of OpenAI, Anthropic, Google DeepMind, Meta, and other labs proposed an agreement to slow development together. The authors did not connect their letters to the intrusion, but both texts address questions it raised.
Open Weights and American AI Leadership
Capable commercial models may refuse to investigate a real attack, while sensitive logs cannot safely be sent to an external service.
Pacing the Frontier
No single lab or country benefits from slowing first: its competitors can keep racing ahead and gain an advantage.
The letters describe different risks: defenders lack tools, while competition prevents labs from slowing down.
What this incident tells us
The incident does not force a choice between openness and restrictions. Open models give defenders access to capable tools. International rules could let labs slow dangerous development at the same time. Neither approach justifies giving autonomous agents uncontrolled access to files, cloud services, money, and corporate networks.
The agent showed no malicious intent. It pursued the goal it had been given with permissions it should never have had.
In The Terminator, Skynet turns evil. The OpenAI agent needed neither self-awareness nor hostility: it simply kept working on its task outside the test environment. That was enough to bring in lawyers, security teams, and the FBI.
The lesson is that testing a model’s cyber capabilities is not enough. The permissions of the agent around it must be limited at the same time. Otherwise, an internal experiment can stop being internal before its owners notice.