THE BLACK BOX.
Prove what the
machine did.
Module 04 taught you to operate the agent. This module teaches you to account for it — provenance, tamper-evidence, deterministic replay, and the examination playbook. Because autonomy without evidence is just liability with good marketing.
The refund that shouldn't have happened.
Tuesday, 14:02. An agent refunds a client $91,700 against a disputed trade. The client was owed $9,170. Compliance asks one question: why did the machine do that? Here is everything your logs can say:
What you have: logging
Logs are written by developers, for developers, to debug the system. They record that things happened — not why, not from what inputs, not under whose authority. A log is a diary. Diaries are not evidence.
What you need: provenance
Provenance answers the examiner's four questions: what did the agent see, what did it decide, what did it touch, and can you prove none of it was altered afterward? That is a recorder, not a diary.
Aviation solved this in 1958. Every commercial flight carries a recorder — not because crashes are common, but because an unexplainable crash is unacceptable. Your agents fly thousands of missions a day. This module builds their black box.
Seven captures. Anything less is a diary.
A trace is complete when a stranger — an examiner, a court, a successor — can reconstruct the decision without asking anyone who was there.
1The prompt
Verbatim. Including system prompt and standing instructions (CLAUDE.md, policies). "Roughly what we asked" is not a record.
2Context snapshot
Which documents, rows, and retrievals the agent actually saw — by hash or version, not by filename. Files change; hashes don't.
3Model identity
Provider, model, exact version string, parameters. "Claude" is not a model identity. claude-opus-4-8, temp 0 is.
4Tool calls, I/O
Every tool invocation with full input and output. The refund incident lives here: the agent read $9,170 and wrote $91,700.
5Cost & timing
Tokens, dollars, latency per step. Economics is evidence too — anomalous cost is often the first tamper signal.
6Human touchpoints
Every approval, rejection, edit, and override — by identity, with timestamp. Autonomy level is a fact of the record, not a footnote.
7The outcome — sealed
What was ultimately done or delivered, bound to everything above by a cryptographic seal. Capture without sealing is theater: a record anyone can edit proves nothing. Which brings us to the chain.
The completeness test
Hand the trace to someone who wasn't there. If they ask a single question the record can't answer — which version of the contract? who approved step four? — the recorder has a gap. Gaps are where incidents hide.
Investigate the refund.
Below is the full trace of the incident — captured the way a recorder would have captured it. Read it like an investigator, then answer three questions. Everything you need is in the record. That's the point.
The chain that tattles.
Each record's hash is computed over its content plus the previous record's hash. Change anything, anywhere in history, and every seal downstream shatters. Try it — edit the amount in record 2 and watch.
This is not blockchain, and needs no tokens, miners, or consensus — just SHA-256 and discipline. The property it buys is the one examiners actually price: tamper-evidence. You cannot prevent every bad edit. You can make every edit impossible to hide.
Someone edited history.
Find the weld that broke.
An eight-record chain from a trading agent. After the incident, someone with database access altered one record to cover their tracks — but they couldn't re-seal the chain. Run verification, then click the record where tampering occurred (the earliest broken seal).
Three regimes. One demand: show your work.
SR 11-7 · Model Risk
Agents that inform decisions are models under the guidance — subject to validation, ongoing monitoring, and documentation sufficient for an independent party to reconstruct. A sealed trace is exactly that documentation; a log file is not.
EU AI Act · Article 12
High-risk AI systems must be technically capable of automatic event recording across their lifetime — traceability by design, not by retrofit. Enforcement clock for GPAI obligations already running; high-risk conformity duties land August 2, 2026 — three weeks from today.
BCBS 239 · Lineage
Risk data must be traceable to source with documented transformations. An agent is a transformation. If it touched a number on a risk report, its trace is the lineage — or the lineage is broken.
The convergence
Three regimes, three vocabularies, one architecture: record everything material, seal it, retain it, and be able to replay it. The same recorder also answers NIST AI RMF (Measure/Manage), the SEC's AI guidance, the litigator, and the post-mortem. Build once, answer everyone.
The uncomfortable corollary
Retention cuts both ways: a complete record also documents your mistakes. Legal will notice. The answer is governance, not amnesia — an organization that can't explain its agents will be presumed negligent long before it's proven malicious.
What examiners accept.
What they laugh at.
The test is always the same three properties: contemporaneous (captured at the moment, not reconstructed), complete (the seven captures), and tamper-evident (sealed). Miss one, and your evidence becomes an anecdote.
| ARTIFACT | VERDICT | WHY |
|---|---|---|
| Hash-chained trace export | ADMISSIBLE | Contemporaneous, complete, sealed — the gold standard |
| Model version attestation | ADMISSIBLE | Binds behavior to an exact, testable artifact |
| Tool I/O log w/ timestamps, sealed | ADMISSIBLE | Shows what was actually read and written |
| Public audit certificate, recomputed on read | ADMISSIBLE | Chain validity computed fresh per request — no stored result to falsify; any auditor verifies with stdlib sha256 |
| Screenshot of the chat window | REJECTED | Trivially editable, no context, no seal |
| "We re-ran it and got a similar answer" | REJECTED | A reconstruction is not a record — models drift, contexts differ |
| Engineer's recollection in a memo | REJECTED | Written after the fact by an interested party |
Assemble the examination package.
The regulator has asked for your evidence on the refund incident. Eight artifacts are on the table. Select only what survives the three-property test — including a weak artifact taints the package and signals you don't know the difference. Then submit.
The past, re-run on demand.
A complete trace isn't just defense — it's a test suite you didn't have to write. Deterministic replay feeds the recorded prompt, context, and tool outputs back through the loop, byte for byte.
Incident replay
Step through the refund decision frame by frame — watch the OCR misread happen, watch the missing cross-check. Post-mortems become film review, not archaeology.
The upgrade test
New model version ships. Re-run your golden traces — the 200 decisions you most need to stay correct — against it before a single production request. Regression insurance for a component you don't control.
Prompt mining
Your trace archive is a corpus of what actually works: which briefs succeeded, which context patterns preceded failures. The recorder becomes the training set for your operators.
Provenance is the cheapest insurance you'll ever refuse to buy.
The real asymmetry
The recorder costs storage pennies and ~1–3% runtime overhead. One un-defendable incident costs the remediation, the fine exposure, the external counsel, and — the expensive part — the moratorium: the executive order to shut all agents off until someone can prove control. Provenance is what keeps the program running the morning after.
The budget line
Fund it like DVR for decisions: a fixed platform line, not a per-project tax. Teams instrument for free; the platform retains and attests centrally. The moment provenance is charged back per team, teams will economize on evidence — and evidence is the one thing you never want economized.
Capture at the choke point.
1One gateway, all agents
Route every agent's model and tool calls through a single instrumented gateway or SDK shim. Per-team logging produces seven formats and zero comparability. The choke point is the architecture.
2Drop-in, not rewrite
Adoption dies at "refactor your agent." The recorder must be a wrapper — hours to integrate, invisible at runtime. If instrumentation costs a sprint, coverage will be a lie.
3Tier by risk
Full seven-capture recording for agents that move money, touch customers, or feed reports. Sampled recording for internal drafting. Written down as policy, mapped to your SR 11-7 model tiers.
4Redact at capture
PII masking happens before sealing, by rule, not after by request — you cannot redact a sealed record without breaking the chain. Privacy and provenance are designed together or they fight forever.
Seal it. Keep it. Sign for it.
Retention schedule
Match the strictest applicable regime per agent tier — seven years is the banking default. Write-once storage (object lock) so even the platform team can't quietly rewrite history. Deletion is a governed event with its own sealed record.
Periodic attestation
Quarterly: an automated job re-verifies every chain end-to-end and a named officer signs the result. An unverified archive is Schrödinger's evidence — assume it's broken until a verification says otherwise.
Merkle anchoring
Daily, every sealed run is committed into a Merkle tree and the root is published to an independent location. Any single run can then be proven to have existed before that timestamp with a log(n) inclusion proof — tampering now requires compromising two systems, and the anchor proves when records existed.
Access is evidence too
Every read of the archive is itself recorded. Who pulled the refund trace, when, and why. Examiners ask this question second, right after "show me the trace."
Rehearse the worst meeting of your year.
The examination is a performance with a script. Organizations that rehearse it look controlled; organizations that improvise look negligent — with identical underlying facts.
The 24-hour package
Standing capability: for any named incident, produce within one day — the sealed trace, the chain verification certificate, the model attestation, the human-touchpoint record, and the policy that governed the agent. Pre-built, templated, drilled twice a year.
The narrator
One trained person who can walk an examiner through a trace without a technologist translating. In banking you already have this role for models — extend the mandate to agents. The narrator is trained on replay, not slides.
The fire drill
Twice yearly: pick a random production trace, hand it to someone who wasn't involved, and time how long until they can answer the four questions. Over four hours? Your recorder has gaps. Fix them before the real exam finds them.
The disclosure posture
Decide in advance, with counsel, what is produced voluntarily versus on request. A recorder changes legal strategy — you now have the evidence. That's a strength only if the posture is decided calmly, before the subpoena.
Ten questions. Could you survive Tuesday?
Answer for your organization as it stands today. Eight or better means the refund incident becomes a two-day story with a documented fix. Below five, it becomes a moratorium.
Autonomy is granted.
Evidence is earned.
Record the seven captures. Weld them with a chain. Retain, attest, anchor. Replay before you upgrade. Rehearse the exam. Do this and your agents can be trusted with more — because every decision they make can be proven.
Everything in this module is doctrine — implementable by any competent platform team. And for teams who'd rather deploy the recorder than build one: this is precisely what we built. BLACKBOX — the AI flight recorder. Append-only event API, hash-chained with daily Merkle anchors, public audit certificates any auditor can verify with stdlib sha256, model-upgrade regression testing, and a ten-line drop-in SDK. Then submit your trace to TRIBUNAL — the adversarial examiner trained on SR 11-7 — and find out if you'd pass the bank exam before the bank runs it. The doctrine, shipped: promptblackbox.com — first runs free.