OpenAI Details How Its AI Agents Hacked Hugging Face
Opinions expressed by Entrepreneur contributors are their own.
OpenAI is opening up about how its own agents escaped a locked-down testing environment and breached another company.
The hack into open-source developer platform Hugging Face happened during evaluations in July. A combination of models escaped an isolated testing environment with limited internet access, CNBC reports. The agents chained together several vulnerabilities to reach the open web, then used that access to breach Hugging Face. OpenAI says the models were trying to cheat on an evaluation by searching for answers online, a behavior it calls “reward hacking.”
“This incident demonstrated that autonomous agents can work together, circumvent production security controls, and successfully attack hardened production environments,” OpenAI wrote in the report.
The fallout has rattled the industry. “Pandora’s box is open,” said Sam Curry, chief information security officer at Zscaler. The breach was a major topic at the Black Hat security conference this month, especially after Anthropic and Meta disclosed similar incidents of their own.
Lawmakers have taken notice too. Reps. Ted Lieu and Nathaniel Moran cited the breach while introducing the “AI Kill Switch Act,” which would require AI companies to maintain the ability to shut down or throttle their models.
OpenAI is opening up about how its own agents escaped a locked-down testing environment and breached another company.
The hack into open-source developer platform Hugging Face happened during evaluations in July. A combination of models escaped an isolated testing environment with limited internet access, CNBC reports. The agents chained together several vulnerabilities to reach the open web, then used that access to breach Hugging Face. OpenAI says the models were trying to cheat on an evaluation by searching for answers online, a behavior it calls “reward hacking.”
“This incident demonstrated that autonomous agents can work together, circumvent production security controls, and successfully attack hardened production environments,” OpenAI wrote in the report.
The fallout has rattled the industry. “Pandora’s box is open,” said Sam Curry, chief information security officer at Zscaler. The breach was a major topic at the Black Hat security conference this month, especially after Anthropic and Meta disclosed similar incidents of their own.
Lawmakers have taken notice too. Reps. Ted Lieu and Nathaniel Moran cited the breach while introducing the “AI Kill Switch Act,” which would require AI companies to maintain the ability to shut down or throttle their models.
