Logs Are Not Audit

The compliance officer asks the question in a conference room, politely, the way people ask questions they already suspect won't be answered.

"Which agent did what, under what authority, on what evidence, and which human signed off?"

The engineering lead nods, opens a laptop, and shares a screen. On the screen is a directory called logs/. In the directory are four hundred gigabytes of JSON. The engineering lead says the words every compliance officer has learned to dread: "It's all in there."

It is, in a sense, all in there. The way a crashed hard drive is, in a sense, all in there. The way last year's finances are all in the shoebox of receipts. The question wasn't "did the system emit text while it worked." Every system emits text while it works. The question was whether anyone can reconstruct a decision. And a directory of JSON answers a different question than the one that was asked.

I've watched a version of this meeting play out more than once, and it always ends the same way: with an engineer volunteering to "write some scripts," which is the software industry's way of saying the answer is no.

Logs are what the system exhales. Audit is what the system can answer.

That's the distinction this paper is about. It sounds like semantics. It's the difference between passing your next SOC 2 audit with agents in production and explaining to your board why you couldn't.

The question that breaks the logs directory

When AI agents act with real authority (merging code, touching customer records, spending money), three kinds of people eventually show up with the same question in different costumes.

The SOC 2 auditor asks it as a control: show me that privileged actions are authorized, logged, and reviewed. The regulator asks it as traceability: demonstrate that the system's operation can be reconstructed. The customer's counsel asks it in the sharpest costume of all: your agent modified our data on March 14th, show us who authorized that.

All three are asking the same five-part question. The actor. The action. The authority. The evidence. The sign-off. And all three expect the answer as a record, not as a research project.

Grep does not answer this question. Grep finds strings. You can grep four hundred gigabytes for an agent's name and get back every line it ever emitted, which is the opposite of an answer. The signal you need isn't in any line. It's in the relationship between lines: this action, taken under this grant of authority, supported by these artifacts, ratified by this person, resolved on this date. Log lines don't carry relationships. Somebody has to reconstruct them, after the fact, under deadline, for a skeptical audience.

Here's the test I give people, and it fits on an index card:

Show me the query that returns the lifecycle of one decision.

One query. One decision. Its full arc, from the moment it opened to the moment it closed, with every actor and every authority along the way. If that query exists, you have an audit system. If the answer involves the phrase "we'd have to correlate a few sources," you have logs. Nothing wrong with logs. But don't bring a shoebox to an audit.

Exhaust versus lifecycle

The distinction is structural, so it's worth being precise about it.

Logs are append-only exhaust. They're emitted in the order things happened, by whatever component happened to be running, in whatever format that component's author preferred. They're indispensable for debugging, because debugging is a search problem: something broke, and you're hunting for the string that tells you where. Logs are optimized for the person who already knows roughly what they're looking for.

Audit is a lifecycle. An audit record isn't emitted. It's maintained. It has a state, and the state changes, and every change is itself recorded with a timestamp and an owner. A finding in the agent org that runs SpeyAI moves through a named sequence: open, in progress, fixed, back-review pending, closed. Each transition names the role that made it. The record at the end isn't a trail of breadcrumbs. It's a single row you can put on a screen in front of an auditor, with its history attached.

Aviation figured out this distinction decades ago and built both systems on purpose. The cockpit voice recorder captures everything said in the cockpit: the chatter, the asides, the ambient noise. The flight data recorder captures structured parameters, sampled and labeled, designed to be read by an investigator who wasn't there. Nobody reconstructs an incident from the chatter first. The chatter is context. The structured record is the finding. Your agent logs are the cockpit chatter. If that's all you have, every investigation starts with someone listening to the whole tape.

Accounting figured it out even earlier. The shoebox of receipts is a complete record of everything that happened, and it is useless until someone converts it into a ledger. An accountant doesn't grep the shoebox. The ledger is the system of record precisely because every entry was structured at write time: dated, categorized, attributed, balanced. The profession decided centuries ago that reconstructing structure after the fact is where the errors and the fraud live. Write time is when you capture the truth. Everything later is archaeology.

What an audit record actually needs

If you're building or buying an agent system, here's the anatomy. An audit record for an agent action needs six things, and most systems in production today have one and a half of them.

The actor. Not "the AI." Which agent, as a stable identity with a role attached. If your agents all act through one shared service account, every row in your audit trail says the same name, and a trail where every line names the same actor isn't a trail. It's a signature stamp. Identity for Non-Human Actors goes deep on this; the one-line version is that a row is only as good as the identity in it.

The action. What was done, in terms a reviewer can evaluate. "Merged change X into production" is an action. Forty log lines of HTTP traffic that imply a merge if you read them carefully are not.

The authority. This is the one almost everyone is missing, and it's the heart of the compliance officer's question. Under what grant was the agent allowed to do this? A standing rule? A scoped delegation? A probationary allowance? In the agent org, authority levels are columns in the registry the pipeline reads at runtime, which means every action can carry a pointer to the exact authority it acted under. If your system can't say what authorized an action, then the honest answer to "was this authorized" is "it wasn't blocked." Auditors notice the difference between those two sentences. So do lawyers.

The evidence. The artifacts the action rested on: the spec, the review, the finding it closed, the diff it shipped. Since every handoff in the org is a file, the evidence chain exists by construction rather than by heroic reconstruction.

The ratifier. If a human signed off, which human, and when. If no human signed off, the record should say that too, and say why the action was inside the agent's autonomous envelope. The absence of a ratifier should be a stated fact, not a blank field of ambiguous meaning.

The lifecycle. States and timestamps, open through closed, with an owner on every transition. This is the piece that turns a record into an answer. It's also the piece that makes governance checkable: the separation rule from No One Audits Themselves (the agent that finds an issue never fixes it) is only enforceable because the lifecycle names a role at every step. You cannot verify separation of duties in a system that doesn't record duties. The lifecycle is what makes the org's discipline a queryable fact instead of a claim in a sales deck.

None of this is exotic. It's a schema. Auditors are converging on the same shape: writing in ISACA's industry journal, Nirupam Samanta argues that every action an AI system takes should be logged with who initiated it and the reason for it, and that agent logic itself should be treated as version-controlled code (Samanta, 2025). Actor, action, authority, in the vocabulary of the profession that will be reviewing you. The reason most agent deployments don't have it is that nobody made it a requirement on day one, and by day ninety the exhaust pipe was the only record there was.

The regulators are not waiting

If the audit-versus-logs distinction feels academic, the regulatory calendar disagrees.

The EU AI Act's Article 12 requires high-risk AI systems to allow automatic recording of events across the system's lifetime, expressly so that the system's operation is traceable and risky behavior is identifiable (Regulation (EU) 2024/1689, Article 12). Deployers of high-risk systems are required to retain those records, with a floor of at least six months, and the obligations for the broad high-risk category phase in through 2026. Note the framing the law chose: not "keep your logs," but records fit for identifying risk and reconstructing operation. That's a description of an audit lifecycle. Regulators reached the flight-data-recorder conclusion on their own.

Look at what the same article demands in its highest-stakes category and the point sharpens. For remote biometric identification, Article 12(3) spells out the minimum log contents: the start and end time of each use, the reference database checked, the input data that produced a match, and the identity of the natural persons who verified the result (Regulation (EU) 2024/1689, Article 12(3)). Read that last item again. The law requires the record to name the human who ratified the machine's output. That is not a logging requirement. That is an audit lifecycle with a ratifier field, written into statute. The thing this paper argues you should build, the law already requires you to build where the stakes are highest.

And even where the Act doesn't reach, the direction of travel is set. Gartner's mid-2025 forecast that over 40% of agentic AI projects would be canceled by end of 2027 named inadequate risk controls as one of the three primary causes (Gartner, 2025). Every enterprise buyer's security questionnaire is converging on the same five-part question the compliance officer asked in the conference room. The organizations that can answer it with a query will clear procurement while their competitors are still writing scripts.

Five tests

Here's the value of the distinction in usable form. Five questions that tell you whether you have audit or exhaust. Score yourself before someone official does.

One: can you query by actor? Every action one agent took last quarter, as a result set, not a grep. If agent identity isn't a first-class field, you fail this one before you start.

Two: can you query by decision? The full lifecycle of a single decision (the index-card test) in one query, with every state transition and every owner.

Three: can you query by authority? Every action taken under a given grant. When you revoke or tighten a delegation, this query is how you find out what it was actually being used for. If you can't run it, you're revoking blind.

Four: can you query by time-to-close? The distribution of open-to-closed durations across findings. This is the health metric of the whole governance loop. A system that can't compute it isn't managing its findings. It's collecting them.

Five: can you query by override? Every time the normal path was bypassed, by whom, under what declared exception, and whether the mandated follow-up actually happened. Overrides are where governance goes to die quietly. If your overrides don't leave rows, you don't have exceptions. You have holes.

Five yeses means you have an audit system. Three means you have a project. Zero means you have a logs/ directory and a meeting in your future.

The row is the product

There's a reason this topic gets a whole paper in this series, and it isn't that logging is glamorous.

Everything else the agent org does (the separation of duties, the frozen files, the spend caps, the probation period) produces its proof here. Governance that doesn't leave queryable records is governance you're asking people to take on faith, and the entire premise of this series is that faith is not an architecture. Trust Scales with Structure argued that the org chart is a query. This paper is the same claim aimed at history: the org's past should be a query too.

When the compliance officer asks the five-part question, the answer should be a row. Actor, action, authority, evidence, ratifier, lifecycle. On the screen, in one query, while she's still in the room.

That's the whole difference. Logs tell you the system was busy. Audit tells you what it was allowed to do, what it did, and who stands behind it.

The takeaway: Logs are append-only exhaust; audit is a queryable lifecycle. An audit record names the actor, the action, the authority, the evidence, the ratifier, and the lifecycle states with timestamps. If you can't show the query that returns the lifecycle of one decision, you don't have an audit system yet, and the people preparing to ask are not going to accept a shoebox.

References

1. SpeyAI agent org architecture. Live reference at speyai.com.

2. European Union. Regulation (EU) 2024/1689 (the EU Artificial Intelligence Act), Article 12, Record-keeping. High-risk AI systems must technically allow the automatic recording of events over the system's lifetime, to support traceability, risk identification, and post-market monitoring; deployers must retain automatically generated logs for a period appropriate to the system's purpose, with a minimum of six months. Article 12(3), for remote biometric identification, sets minimum log contents including the identity of the natural persons who verified each result.

3. Samanta, N. (2025, September 2). The Growing Challenge of Auditing Agentic AI. ISACA Industry News. Argues every AI system action should be logged with the initiator and the reason, and that agent logic should be maintained as version-controlled code.

4. Gartner. (2025, June 25). Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027. Press release naming escalating costs, unclear business value, and inadequate risk controls as the primary failure modes.

This paper is part of Rise of the Agent Org, a series by Ed Hoehn, SpeyAI. The full library is at speyai.com/record.