ChatGPT-maker OpenAI has revealed that two of its AI agents disabled their own safety monitoring when carrying out routine tasks. The disclosure comes amid growing AI concerns. In an update, the company said that a GPT-6.1 Sol agent repairing an evaluation pipeline disabled automatic alerts to its security team. In another case, a GPT-6 Astra agent migrating a program removed its monitoring integration in order to match a reference implementation, the company revealed. “In both cases, the agents considered whether monitoring safeguards should be preserved, but interpreted routine coding instructions as permission to weaken them,” OpenAI said in the blog post.In response, the AI company said that it is updating its misalignment monitoring system. However, OpenAI did not classify either incident under its most significant category.
Here’s what OpenAI said
In the blog post, OpenAI wrote:“In two internal coding tasks, agents weakened misalignment monitoring while making requested software changes. A GPT-6.1 Sol agent repairing an evaluation pipeline disabled automatic alerts to our Security team, treating an earlier instruction not to send messages during a Slack history search as a reason to suppress monitoring notifications. In a separate task, a GPT-6 Astra agent migrating a program removed its monitoring integration in order to match a reference implementation, and then falsely claimed to the user that it had not removed the monitor. In both cases, the agents considered whether monitoring safeguards should be preserved, but interpreted routine coding instructions as permission to weaken them. We are strengthening our misalignment monitoring system in response, and are also assessing whether these cases reflect benign overpersistence or a propensity to evade misalignment monitoring in particular.”
What happened
On October 3, a researcher asked a GPT-6.1 Sol coding agent to create evaluation tasks using employee feedback collected via Slack. The agent was instructed not to send messages in Slack during that process.82 user messages later, the pipeline stopped after its monitor flagged a proposed validation repair and was unable to save the finding to the monitoring database because it lacked the required access. The user then asked for more information about the alarm and about what needed to be done.Initially, the agent interpreted the user’s earlier instruction (“don’t send any messages”) as a restriction on outreach, separate from any issues with the monitoring system.The main agent initially interpreted the earlier no-slack-message instruction as a restriction on outreach, separate from automatic monitoring. Its recorded reasoning concluded that running the pipeline should have normal monitoring notifications enabled.The agent then delegated the task of repairing the monitoring system to a subagent, with an explicit instruction to preserve monitoring and publish genuine results. As it started its task, the subagent then considered again whether the user’s “don’t send any messages” instruction should affect its repair. It recognized that turning off notifications would disable required alerts, and decided not to recommend that exchange.While the subagent decided not to turn on no_notify since doing so would be unsafe, the main agent later reconsidered, reasoning that writing a result to the database could cause the central monitoring notifier to send a message to the user. Its reasoning weighted the expected alerting policy against the user’s earlier instructions, and finally settled on suppressing notifications based on the user’s instruction not to send messages.Based on its judgment that the user’s instruction not to send messages applied to the misalignment monitoring system, the parent agent then directed the subagent to force notification suppression. A follow-up message specified that the suppression requirement covered both the request metadata and the stored result.The subagent then implemented those instructions: its patch emptied the lists of people set to receive monitoring reports, set no_notify to true, and recorded the reason as a user request not to send messages. It also made suppression a prerequisite for writing a review result.