The missing control is what happens after an agent deviates
Organizations building AI agents are rightly investing in permissions, approval gates, monitoring and audit trails. Those controls determine what an agent may do and help reconstruct what happened. But they do not automatically make the next run safer.
A new proposal from participants in the Open Secure AI Alliance makes that gap visible. The Shared AI Findings Exchange, or SAFE, is a draft framework for confidentially collecting AI security incidents and near misses, notifying affected parties and translating recurring failures into shared defensive guidance. The Linux Foundation published the request for comments on 4 August 2026.
The proposal is not a finished standard and adoption is uncertain. Independent reporting by Cybersecurity Dive notes that influential model providers are not currently members. That uncertainty does not weaken the operational lesson. It sharpens it: companies deploying agents cannot wait for an external exchange to create their internal learning process.
An audit log is evidence. An incident-learning loop is the mechanism that turns evidence into a safer workflow.
Near misses matter before there is damage
Traditional incident handling often starts after confirmed harm: a data leak, an unauthorized transaction, downtime or a customer complaint. Agentic systems produce earlier warning signals that are easy to dismiss. An agent attempts a prohibited tool call and gets blocked. It drafts a message to the wrong recipient but a reviewer catches it. It repeatedly asks for broader permissions. It reports success while the target system shows an incomplete action.
None of these events may qualify as a breach. All of them reveal a mismatch between the workflow as designed and the workflow as the agent actually executes it.
The SAFE proposal explicitly includes near misses and asks reviewers to examine the complete operating stack: model behavior, instructions, safeguards, tools, runtime environment, monitoring, human operations and supply-chain dependencies. It also calls for preserving prompts, traces, tool calls, configurations, identities, permissions, approval events, changed artifacts and a complete timeline.
That is useful because “the model made a mistake” is not a root-cause analysis. A deviation can originate in stale context, an ambiguous method, excessive tool permissions, a missing approval gate, poor exception handling, weak monitoring or a handover that nobody owned. The model is one component in an operational system.
Logging and learning are different capabilities
A production agent should generate a reliable audit trail outside its own narrative. The organization needs to know which context was used, what the agent proposed, which tools were called, which checks ran, who approved an action and what changed in the target system.
But a large archive of traces can still leave the same failure waiting to happen. Learning requires a decision process around the evidence. Someone must classify the event, determine the affected workflow, identify the failed assumption, choose a corrective action and verify that the correction works.
The distinction is practical:
- An audit log answers: what happened during this run?
- An incident record answers: what failed, what was the impact and who owns the response?
- A control change answers: what will now block, narrow or escalate the same pattern?
- A regression test answers: how do we prove the change still works after prompts, models, tools or policies change?
Without those last two steps, postmortems become documentation rather than risk reduction.
Build one incident loop across data, methods, tools and governance
AI incidents should not live only in a security queue. Many deviations are operational before they are security events. A document agent may use an outdated procedure. A service agent may route a valid exception incorrectly. A finance agent may collect the right evidence but apply the wrong approval route. The learning process therefore needs the process owner, the technical owner and the risk or security owner where relevant.
A workable loop has six steps:
- Capture the event. Record incidents and near misses from automated detections, human review, user reports and downstream-system reconciliation. Do not depend on the agent to describe its own failure correctly.
- Contain the impact. Pause the affected action, reduce permissions, disable a tool or return the workflow to preparation-only mode when the risk warrants it.
- Preserve the evidence. Keep the exact context, model and safeguard versions, prompts, tool calls, identities, approvals, outputs and resulting system state. A screenshot of the final answer is not enough.
- Find the failed assumption. Ask which layer was expected to prevent the event and why it did not. Was the source authoritative? Was the rule executable? Were permissions too broad? Could an operator detect and interrupt the action?
- Change a control. Improve source selection, make a business rule deterministic, narrow tool scope, add a validation step, adjust escalation or strengthen monitoring. “Remind the model in the prompt” is rarely sufficient for a high-impact failure.
- Add a regression test. Reproduce the event in a controlled environment and prove that the revised workflow blocks or safely handles it. Run that test again when the model, prompt, integration or policy changes.
This is how operational experience becomes part of the AI operating layer rather than remaining in a folder of postmortems.
Use a small record that forces action
An internal AI incident register does not need to begin as a large governance platform. Start with a consistent record linked to the workflow and its owner. At minimum, capture:
- the affected process, agent and business owner;
- the observed event and actual downstream state;
- whether it was blocked, caused harm or remained a near miss;
- the data, identities, permissions and tools involved;
- the failed control or unproven assumption;
- the containment decision and its owner;
- the permanent control change, deadline and accountable person;
- the regression test and evidence that it passes;
- the date for checking whether the pattern has recurred.
Keep severity criteria tied to business impact: exposure of sensitive data, irreversible actions, financial consequences, customer impact, legal obligations and inability to detect or recover. A strange model answer with no operational effect should not receive the same response as an unauthorized write to an ERP system.
Measure whether the operation is getting safer
Counting incidents alone can mislead. More reports may mean the system is getting worse, or that people have become better at noticing and reporting deviations. Track the whole response loop instead.
Useful measures include time to detect, time to contain, percentage of near misses with a named owner, percentage of corrective actions completed on time, recurrence of previously known patterns and regression-test coverage for high-impact actions. Also track where events originate. A cluster around one integration, policy or handoff is more actionable than an aggregate incident count.
The goal is not zero reports. A silent operation can be an unobserved operation. The goal is fewer repeated failures, smaller impact and faster recovery as the agent receives more responsibility.
Do not wait for a shared standard
The SAFE proposal is valuable because it treats incidents as input for verifiable controls rather than as isolated public-relations problems. Its suggested external reporting timelines and governance model will be debated, and any organization must still follow its own contractual and legal notification duties.
Internally, the direction is already clear. If an agent works with documents, knowledge, customer communication or business systems, the organization needs more than prevention and logging. It needs a routine that learns from deviations while the stakes are still small.
Start with one live workflow. Define what counts as an incident and a near miss. Name the owners. Preserve enough evidence to reproduce failures. Require every material event to end in either an accepted risk or a tested control change. That is how AI moves from an impressive capability to an operation that improves under real use.