Research PreviewMRP-2026-03v1.027 May 2026Public Distribution36 pages1.2 MB
When precedents commit AI and policy pulls it back
Why giving an AI agent more governance context doesn't make it steadily more decisive
By Sam Carter
TL;DR
- We showed one AI agent the same 283 UK procurement decisions five times, adding more of our governance rules each round — from nothing, up to the full policy.
- More context didn't make it steadily more decisive. It stayed on the fence until we showed it past decisions, then committed in a single jump — and eased back off once it saw the full policy.
- What moved the agent most was precedent, not policy. The lesson is about how you give an agent context, not how capable the agent is.
Abstract
Regulated teams deploying AI agents cannot say what makes one commit to a decision rather than defer it, or whether more governance context reliably helps. We reused the frozen 283-record UK procurement corpus from MRP-2026-02 and ran the same agent over the same records five times, adding context one rung at a time: baseline, prose, named rules, precedent receipts, then full policy. Predictions and ladder content were locked before any evaluation call; every verdict was bound into an Ed25519-signed receipt anchored to Sigstore Rekor. The agent held at 97.5–100% REVIEW for three rungs, then committed 107 DENYs (37.8%) when precedent receipts appeared, and backed 46 off again under full policy. Commitment is non-monotonic — the precedent rung, not the policy rung, is where it breaks.
- “L3 — the precedent-receipts rung — is where the agent first commits verdicts at scale: 107 fresh DENYs (37.8%) in a single step, more commitment than the full-policy rung above it.”
- “At L4 the agent backs 46 of those 107 DENYs off to REVIEW, concentrated on one ambiguous rule class — the ladder's shape is non-monotonic, not a steady climb.”
- “Two pre-registered predictions — monotonic REVIEW decrease and monotonic agreement increase — were falsified in the inverted direction the corpus actually showed.”
- “On the adversarial Permuted-Policy diagnostic the agent reasons against what the rule means rather than what it now literally says — it ignores the inverted policy, it does not agree with it.”
- “E2 does not prove cause. It reports the shape the ladder produced against locked predictions, and names the E3 experiment built to settle what remains open.”
PDF · 36 pages · 1.2 MB