Why an approval queue for AI agent edits fails, and what works

An approval queue in front of AI agents' writes fails twice. Agents change files with their own Write and Edit tools, so an MCP "propose an edit" tool is an optional path that gets bypassed. And a gate that caught everything would, on a vault where 303 pages changed in a week, become a queue cleared with "accept all". Lore replaced its queue with a filesystem watcher that journals every change, a triage that ranks changes by lines deleted, inbound links and sign-off state, and sign-offs pinned to a content hash, so an agent's rewrite cancels them.

Symptoms

Lore v0.1 (commit 51ba865, 26 July 2026) had no MCP write tool, only propose_edit (trimmed from mcp/server.mjs):

{
  name: "propose_edit",
  description:
    "Propose a change to the wiki. This does NOT write the file — it queues a diff for the human to accept or reject. ...",
  inputSchema: { /* path, kind: create | append | replace, content, reason, risk */ },
}
// handler: POST /api/proposals, then
return `Proposed. ... The file has NOT been changed. Do not assume it was accepted.`;

The next commit, two hours later (dd5e39a), recorded what testing found: "a direct Write bypassed propose_edit and Lore never noticed."

The volume figures come from the 1,424-page vault Lore was built against (docs/BUILD-LOG.md):

• 303 pages changed in seven days, 56 of them in the last 24 hours.
• 60 of 75 modified files were rewritten in place rather than appended to.
• 1,156 pages carried an agent-written confidence: field: 83% said high and none said low.
• No page carried a human reviewed or verified field.

Why it happens

The gate sits on one path and the writes take another. An agent may choose to call an MCP tool, but its built-in file tools write straight to disk, and nothing on the filesystem routes through the MCP server. The v0.1 comment "propose_edit is the only way an agent can change anything" was true of the MCP surface and false of the machine.

Volume finishes the job. The build log: "303 changes/week through a gate is a 303-item queue, which resolves to 'Accept All' and manufactures confidence — worse than no gate."

The fix

The gate was deleted from the MCP surface in dd5e39a, then from the rest of the code in bb58841, which found POST /api/proposals could still write into the vault with no caller left. The replacement is in lib/journal.ts, lib/trust-core.ts and app/api/review/route.ts.

Watch the folder, and trust hashes over events

const watcher = chokidar.watch(root, {
  ignored: (p) => /(^|[/\\])(\.git|\.obsidian|\.trash|node_modules)([/\\]|$)/.test(p),
  ignoreInitial: true,
  awaitWriteFinish: { stabilityThreshold: 400, pollInterval: 100 },
});

const onChange = async (abs: string, removed = false) => {
  if (!/\.mdx?$/i.test(abs)) return;
  const relPath = path.relative(root, abs).split(path.sep).join("/");
  const after = removed ? null : await fs.readFile(abs, "utf8").catch(() => null);
  const before = await readShadow(key, relPath); // the last copy Lore saw
  if (after !== null && before !== null && hashOf(after) === hashOf(before)) return;
  const { kind, linesAdded, linesRemoved } = classify(before, after);
  await appendEvent(key, { at: Date.now(), relPath, kind, linesAdded, linesRemoved,
    hash: after === null ? "" : hashOf(after) });
  // ...snapshot the old text, then update the shadow copy
};
// and reconcile(root) every 90 s: re-hash every page against its shadow copy

Watching the filesystem catches Claude Code, Cursor, a shell script and a person typing, with nothing opting in. An event only prompts a re-read and re-hash, because file events are not a reliable record. With awaitWriteFinish, chokidar holds add and change events "until the size does not change for a configurable amount of time" (chokidar README), and Node's docs warn that fs.watch "is not 100% consistent across platforms" (Node.js fs caveats). The 90-second reconcile picks up whatever the events miss.

classify() compares the new text with the shadow copy. If every old line is still at the top, in order, it is an append; anything else is a rewrite, with lines counted outside the common prefix and suffix. Appends never remove a line.

Rank the week instead of queueing it

const deletionWeight = e.linesRemoved * 3;
const additionWeight = e.linesAdded * 0.2;
const reachWeight = Math.log2(inbound + 1) * 8; // pages linking here
const trustWeight = trust === "lapsed" ? 25 : trust === "unverified" ? 10 : 0;
const rewritePenalty = e.kind === "rewritten" ? 12 : 0;

Repeated writes to one page collapse into its most severe edit. The review screen shows the top 12 with a plain reason, such as "12 lines removed · 20 pages link here · you had verified this". Additions to pages nothing links to score low, and that is most of the weekly volume.

Pin the sign-off to the content

export function trustOf(ledger, pageId, currentHash, now = Date.now(), decayDays = DECAY_DAYS) {
  const v = ledger[pageId];
  if (!v) return "unverified";
  if (v.hash !== currentHash) return "lapsed";
  if (now - v.at > decayDays * 86_400_000) return "aging";
  return "verified";
}

The hash covers the plain-text body, so an agent bumping updated: in the frontmatter doesn't lapse a sign-off, but any prose change does. The ledger lives in ~/.lore, outside the vault. wiki_read in the MCP server tells the next agent: "LAPSED — a human confirmed an earlier version, and it has been rewritten since. Treat as unverified."

How to check you've fixed it

Write the way agents do, with a plain file write rather than your tool, and the change should still show up. We ran Lore's real journal.ts and trust-core.ts against a scratch vault, with HOME redirected so the journal landed there too:

• An append, a rewrite and a new page each produced one journal event: appended +40 -0, rewritten +1 -12 and created +61 -0.
• Twenty writes to one file, 10 ms apart, produced one event with the net change: awaitWriteFinish waits for the burst to settle, and the handler diffs against the shadow copy.
• A hub page with 20 inbound links, signed off the day before, went from verified to lapsed after the rewrite and ranked first with a score of 108. The new 61-line page with no inbound links scored 22, and the 40-line append scored 18.

Also test a read straight after a write. The build log records a page that was verified, then rewritten, and still said "verified", because trust came from a cached scan. Now the watcher invalidates the scan cache on every write, and the review endpoint re-indexes before hashing.

Caveats

• A sign-off hashes what is on disk when you click, not what you read. The review screen sends only { action, pageId } and the server hashes the current file, so an agent's rewrite between your read and your click gets your signature. Sending the hash of the version you were shown, and refusing on a mismatch, would close that gap.
• Coverage starts when watching starts. The review, changes and history endpoints start the watcher lazily. Edits made while Lore was closed arrive through reconcile() as one net event per file. In our test, a rewrite made while nothing was watching was picked up (rewritten +0 -5), but a file deleted in that window was never journaled, because reconcile() only walks files that exist.
• The weights are hand-set constants, with no calibration run in the repo. An aging page scores the same as a verified one.
• The journal is anonymous. Lore's optional Claude Code PostToolUse hook (matcher Write|Edit|MultiEdit, lib/harness.ts) adds attribution for that one harness.
• Versions. The quoted code is on the public repo's main: chokidar 5.0.0, Next.js 16.2.2, Node 20.9+. We tested on Node 22.