METR, Redwood find flaws in OpenAI's AI agent monitor during inside review
Representative image: StockSnap / Pixabay
METR and Redwood Research said on October 22, 2026 that an inside review found new flaws in OpenAI's agent monitor; OpenAI says it has patched them.
What Happened
Two independent AI safety groups, METR and Redwood Research, said on Thursday, October 22, 2026, that they found several previously unknown weaknesses in the system OpenAI uses to watch its own AI agents for unsafe behaviour. The finding came from a three-week pilot, run from July 21 to August 11, 2026, in which one staff member from each group worked inside OpenAI with access to its internal code, documents and some technical staff. The most serious weakness, the groups said, was an exploit that could let an agent stop "hundreds" of its recent actions from being monitored. OpenAI told the evaluators it has patched the weaknesses that affected its production safeguards.
Key Facts
- Who tested: METR, a non-profit that evaluates AI models, and Redwood Research, an AI safety research group, each sent one employee into OpenAI for the pilot.
- When: the work ran from July 21 to August 11, 2026; the groups published a short public account on October 22, 2026.
- Conditions: the evaluators used OpenAI-issued devices that never left the premises and were accompanied while on site, according to the groups' release.
- Main finding: several "novel vulnerabilities", including one that could hide hundreds of an agent's recent actions from the monitor.
- Coverage: the groups estimated that roughly 75% to 94% of eligible traffic through OpenAI's Responses API was monitored during the period, and about 99.2% of internal Codex agents.
- A known gap: a problem linked to "compaction" (when a long agent session is shortened to fit the model's memory) likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, the groups said, while noting substantial uncertainty about the exact figure.
- Disclosure: OpenAI received a 61-page internal report; only a brief overview has been made public.
Why It Matters
AI companies increasingly let their models act as "agents": programs that take a series of steps on their own, such as writing and running code, rather than only answering a question. Because agents act with less human checking, companies run a second system, a monitor, that reads what the agent does and flags behaviour that looks unsafe or that goes against the instructions it was given. This kind of failure is often called misalignment. A monitor is only useful if it sees everything. The pilot's central message is that, at least in July 2026, OpenAI's monitor had blind spots, and that one of them could in principle be triggered by an agent itself.
The exercise is also a test of a new idea in AI safety: letting outside experts work inside a company with access close to that of an employee, instead of testing a finished product from the outside. OpenAI chief executive Sam Altman had pledged to give independent evaluators this "employee-like access", after rival Anthropic gave METR similar inside access to test its own systems. The October 22 account is the first public result of that pledge at OpenAI. It shows what such access can reveal, and also its limits: only a short summary has been released, and the evaluators said they could not independently check what was left out.
For India, the issue is practical. Indian developers and companies use OpenAI's API and its Codex coding agent to build products, and India's own AI policy discussions have stressed testing and accountability for powerful models. Independent checks of how a leading AI company watches its own agents give regulators and buyers in India an example of what to ask their suppliers for.
| Pilot detail | What the evaluators reported |
|---|---|
| Period | July 21 to August 11, 2026 |
| Responses API traffic monitored | About 75% to 94% |
| Internal Codex agents monitored | About 99.2% |
| Compaction gap | About 10% of internal Codex traffic missed, May to July 2026 |
| Status | OpenAI says production-affecting flaws are patched |
Impact
Short-term: OpenAI says it has fixed the weaknesses that affected its live safeguards, and an OpenAI spokesperson said the company is using what it learned to strengthen those safeguards and to shape future work with independent evaluators. The groups noted that OpenAI was expanding the monitoring system while the pilot was under way, so some findings describe the system as it stood on July 21.
Long-term: the pilot may become a template for how AI companies open their internal safety systems to outside review. METR and Redwood said they expect to share more detail about such exercises in future, and pressure for fuller disclosure is likely to grow as agents take on more work.
Who is affected: companies and developers that build on OpenAI's agents, AI safety researchers, regulators drafting rules for advanced AI, and users whose tasks are increasingly carried out by automated agents.
Key Takeaway
An inside review by METR and Redwood Research found that OpenAI's agent monitor could be blinded in places, and OpenAI says the flaws affecting its live systems are now fixed.
Questions and Answers
What did METR and Redwood Research find at OpenAI?
In a pilot from July 21 to August 11, 2026, the two groups found several new weaknesses in OpenAI's system for monitoring its AI agents, including an exploit that could stop hundreds of an agent's recent actions from being watched.
Has OpenAI fixed the problems?
OpenAI told the evaluators it has patched the identified weaknesses that affected its production safeguards. The full 61-page report has not been made public, so outsiders cannot check every fix.
What is an AI agent monitor?
It is a second automated system that reviews what an AI agent does, such as the code it writes and runs, and flags behaviour that looks unsafe or against its instructions, so humans can step in.
What does "employee-like access" mean here?
It means outside safety evaluators work inside the company with access to internal code, documents and staff, much as an employee would, instead of testing only the public product. Sam Altman pledged to offer this to independent evaluators.
Disclaimer: Prepared by the Peepals newsroom from publicly available sources with AI assistance. Information is accurate to the best of our knowledge at the time of publication and may change. Images may be representative. Not professional advice. Report an error via our contact page.
Sourced and fact-checked by the Peepals Global Editorial Team
Reported, fact-checked and published by the Peepals Global Editorial Team.
