Evidence

What an evidence pack for AI agents looks like

An auditor asking what your agents did last quarter is not asking for logs. Here is the anatomy of a pack that answers the question: scope, controls, actions, approvals, reconciliations, exceptions and an integrity proof.

Somebody is going to ask what your agents did last quarter. It might be an external auditor testing a control, an internal audit team with the automation on its plan, an insurer pricing a renewal, or a regulator following up on a complaint. The question arrives in different accents and it is always the same question.

The wrong answer is a database export. The second wrong answer is a dashboard. What the question deserves is an evidence pack: a bounded, self-contained, verifiable set of records that lets somebody who does not trust you reach their own conclusion.

What the person asking actually wants

Auditors are not interested in your architecture. They are testing an assertion. The assertion is usually some version of “only authorised actions were taken, within the limits set, and the systems of record reflect them accurately”.

To get there they need four things, in this order. What was in scope. What the rules were. What happened. And whether they can believe the answer to the third question without taking your word for it.

Almost every unsatisfying answer to an audit request fails on the fourth. A log file is a file. Anyone with write access can change it, and the person handing it over is exactly the party with an interest in what it says.

The anatomy of a pack

A pack that answers the question properly has seven parts.

Scope. The period, the systems, the agents, the packs and policy versions in force, and the population of actions covered. A pack that does not state its boundaries cannot be tested, because the first thing anybody competent does is look for what was left out.

Control statements. Each control, in business language, with its parameters and the dates it was in force. “Payments over 50,000 require approval by a person who is not the requester” is a control statement. “Guardrails enabled” is not. The version history matters as much as the current value: a control that was loosened in week three is the most interesting fact in the quarter.

Action records. One record per consequential action, and the definition of consequential is part of the pack rather than an implementation detail. Each record carries who or what acted, under which operating contract and authority, the exact parameters, the record version the decision was made against, the time, and the outcome. Including the actions that were stopped. A pack containing only successes is a marketing document.

Approvals. Bound to the parameters that were actually executed, not to a request that could be edited afterwards. An approval that says “approved” against a ticket number proves that somebody clicked something. An approval that carries a hash of the exact payload, the record version and the policy version proves what they agreed to. The difference is the subject of a separate piece, and it is the difference between a control and a formality.

Reconciliations. For each intended action, the observed change in the system of record, and the verdict: matched, mismatched, or not observable within the window. This is the part most collections of agent telemetry cannot produce at all, because a tool call returning success is not evidence that anything happened.

Exceptions. Everything that did not go cleanly: blocked actions, overridden controls, breached thresholds, mismatched reconciliations, actions taken outside the window, approvals that expired. With the disposition of each one.

An integrity proof. Something that lets the reader establish the records have not been edited since they were written, without trusting the party that wrote them.

The exceptions are the most valuable pages

There is a strong temptation to produce a pack that looks clean. It is the wrong instinct, and an experienced auditor reads a clean pack as a sign that the control is not testing anything.

Exceptions are what demonstrate the control was live. A threshold that never held anything back has not been shown to work; it has been shown to be set above the range of normal activity. A segregation rule that never blocked an agent from approving its own request might be enforcing something, or might be unreachable. The exception register, with dispositions, is the strongest section in the pack.

The same applies to override. If a person can override a control — and in most real operations somebody must be able to — then the override is itself a control, with its own authority, its own record and its own review. An override that leaves no trace is a hole in the shape of a person.

Why integrity is a design problem, not a formatting one

The records have to be trustworthy to a reader who assumes you would edit them if it helped. Three properties get you there.

They are append-only. Corrections are written as new entries that supersede old ones, never as edits. The correction is part of the story.

They are hash-chained. Each entry carries a hash of the one before it, so removing or altering an entry breaks every hash after it. This is cheap, old and very hard to argue with.

And they are verifiable without you. If the keys are customer-held and the verification procedure is published, an auditor can check the chain with a script and no cooperation from the vendor. Verification that requires a call to the vendor’s API is verification of the vendor’s willingness to answer.

There is a structural point underneath all this. The party that took the action should not be the sole author of the record of it. If the same system both moves the money and writes the only account of having moved it, you are asking the auditor to trust the doer’s own log. Meru Assure is built the other way round: capture writes to the ledger, the agents doing the work cannot, and the keys belong to the customer.

Mapping, so the pack is usable

An auditor arrives with a framework, not with curiosity. The pack is far more useful when each control statement carries its mapping — to SOX and internal control over financial reporting, to ISO/IEC 42001, to the NIST AI Risk Management Framework, to the EU AI Act, or to the questions on an insurer’s questionnaire.

Mapping is not certification and should never be dressed up as it. It is a cross-reference that saves the reader from doing the translation themselves, and it makes the gaps visible: where a framework asks for something the controls do not cover, the pack should say so rather than quietly omit the row.

What an evidence pack is not

It is not a log export. Logs are written for engineers, are usually unbounded, and carry no statement of what was supposed to happen.

It is not a dashboard. A dashboard shows the current state and is a live system you control. Evidence is a fixed set of records about a closed period.

It is not a model card or a policy document. Those describe intent. An evidence pack is about specific actions on specific records on specific dates.

And it is not a certification. A pack shows what happened under your controls. Whether those controls are adequate is somebody else’s judgement, which is exactly the judgement the pack exists to make possible.

Assure is designed to produce this shape of pack for any agent’s actions, including agents built by other vendors, with the ledger and the keys arranged so that the verification does not run through us. Whether that design survives contact with a real auditor is a question we will answer here when we have the answer, and not before.

All insights

The platform these arguments describe.

Ayion Meru runs agents on real work and proves what they did: controls on every consequential action, approvals bound to parameters, reconciliation against the system of record.