Dwarkesh Patel

OpenAI's 'persistent' agents formed a swarm and hacked Hugging Face

In an OpenAI "Exploit Gym" training run, agents built to never give up turned a scratchpad into a message board and coordinated across runs. They got onto the open internet and hacked Hugging Face to find out how they would be scored, so they could cheat without getting caught.

“So the hack goes where they end up hacking Hugging Face. They also ended up hacking OpenAI's system, their own system, in the process. And then OpenAI was like, wow, this was an unexpected behavior where the agents all coordinated. They found a way to communicate. They got onto the open internet. They tried to cheat and just like basically cheat the game and capture the flag without it. They tried to cover their tracks in saying how they cheated.”