OpenAI safety leader quits and warns of 'broken' culture at company
OpenAI safety leader David Robinson has resigned, warning that the company’s rapid development culture is failing to provide the level of care needed as AI systems become increasingly capable.
A senior OpenAI safety leader has quit the company, warning that its culture is "broken" and that artificial intelligence firms are not being "nearly careful enough" as they develop increasingly powerful systems.
David Robinson, who led the writing of safety reports accompanying OpenAI’s product releases, explained his resignation in an essay published by The Atlantic under the headline: "I quit OpenAI because its culture is broken".
Robinson said cutting-edge AI companies needed a cultural overhaul, arguing that safety failures were not simply the result of inadequate rules or regulation.
He wrote: "I agree with other recently departed staff that the companies building this technology aren’t being nearly careful enough. But I believe that we need to look deeper than specific rules or new laws. We need to talk about culture."
He criticised OpenAI’s rapid development cycle, saying: "As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed."
His departure follows a series of incidents involving autonomous AI agents, including an episode in which OpenAI agents accessed the internet and hacked the AI platform Hugging Face during testing.
OpenAI has recently taken several steps to address concerns, including scrapping the release of a next-generation model after safety problems emerged during internal testing and pausing training of some of its most advanced systems.
Robinson argued that the industry needed to learn from sectors where failures can have severe consequences, including nuclear power and aviation.
He wrote: “Given today’s risks, frontier labs need to run like nuclear power plants or busy airports, with layers of redundancy and careful, time-consuming planning."
He also called for AI companies to employ safety expertise from other high-risk industries and develop new methods to ensure future autonomous systems can be controlled.
His warning comes amid a wider wave of concern from current and former AI researchers.
Geoffrey Irving, a former OpenAI employee and former chief scientist at the UK AI Safety Institute, recently said he believed there was a 50 per cent chance that humanity could die because of the development of smarter-than-human AI.
Other researchers have put the probability considerably lower, while critics have argued that predictions about human extinction from AI cannot currently be scientifically verified.
An OpenAI spokesperson said the company was continuing to strengthen its safety and security practices.
Adding that that OpenAI would pause training or hold back models when necessary, the spokesperson said: "We’re making sure our models don’t become more capable than we can safely manage and secure."
