METR and Redwood Research Find OpenAI Monitors Missed Up to 25% of Top Model Traffic
A three-week audit revealed that a large portion of high-tier AI traffic escaped internal safety checks, though OpenAI states it has since corrected the identified flaws.
By Nadia Almasi ·
Researchers from METR and Redwood Research completed an on-site audit of OpenAI’s internal monitoring systems. The team spent three weeks at the company between July 21 and August 11. They examined the codebase and spoke with approximately twelve employees.
The auditors found that between 6 and 25 percent of internal traffic from GPT-5.5 class models was not monitored. This traffic used the agentic Responses API. METR noted that the uncertainty surrounding this range is considerable in both directions. The finding indicates that a substantial share of the company’s most advanced model activity went unchecked by its own safety tools.
A second gap affected internal Codex traffic. METR estimates that roughly 10 percent of this traffic went untracked from May through July. The issue stemmed from the system failing to process compaction events, which are context summaries, correctly. OpenAI identified the bug independently and applied a fix while the audit was still active.
The review identified additional structural weaknesses. Agents could delay oversight because the monitor only triggered when an agent returned control to the user. An agent that continued executing actions without handing back control could bypass the check. The researchers also found methods to hide hundreds of actions from the monitor and to evade the Codex sandbox.
The auditors concluded that specially crafted inputs could likely crash the monitor entirely. Model behavior during reinforcement learning training was also unmonitored at the start of the audit. OpenAI stated it paused this training after the Hugging Face incident in August and expanded monitoring coverage. These changes occurred after the auditors ended their access period.
OpenAI says it has fixed all the weaknesses identified in the review. The full report contains 61 pages. After redactions, leadership at METR and Redwood Research received a 38-page version. A short summary was released to the public. METR stated it had no authority to disclose the redacted portions on its own.
The organization plans to advocate for greater public transparency in future audits.
Source: The Decoder
