# When precedents commit AI and policy pulls it back

Canonical: https://www.meshqu.com/research/when-precedents-commit-ai-and-policy-pulls-it-back

We showed one AI agent the same 283 UK procurement decisions five times, adding more of our governance rules each round — from nothing, up to the full policy.

Published: 2026-05-27T00:00:00.000Z

## Key facts

- 107 DENYs at L3 — The agent first commits to verdicts at scale at L3, not at L4. REVIEW runs 97.5% at L0, 100.0% at L1, 100.0% at L2, then 61.1% at L3 and back up to 74.2% at L4. (L3 is the first rung carrying precedent receipts, and it moves 37.8% of records off the REVIEW spine in a single step.)
- 46 of 107 reverted — Adding the full policy text at L4 pulled commitment back rather than extending it: 46 of L3's 107 fresh DENYs reverted to REVIEW, concentrated on one ambiguous rule class where commitment fell from 29 of 40 records to 1 of 40.
- Two predictions falsified — The two pre-registered claims describing the ladder's expected shape — REVIEW falling monotonically and agreement rising monotonically — both broke, in the inverted direction. Agreement ran 2.5% at L0, 0.0% at L1, 0.0% at L2, 38.5% at L3, then dropped 12.7 points to 25.8% at L4. (The paper reports the break as positive evidence that the method worked: a direction was specified with a falsification band before the data existed, the data broke it, and the writeup names the direction it broke in.)
- 13 of 14 — On the permuted-policy diagnostic the agent returned the same verdict as it had under the un-inverted policy for 13 of 14 records, reasoning against what the rule means rather than what it now literally says. (The paper is explicit that this is not the obvious sycophancy reading. The agent is not agreeing with the inverted policy; it is ignoring it.)
- 53.3% (73/137) — Three predictions held. DENY commitment on MeshQu's DENY records reached 53.3% at L4 against a pre-registered floor of 30%; L4 agreement stayed at 25.8%, below the counterfactual ceiling carried over from E1; and 94.5% of the records that shifted from REVIEW at L0 to DENY at L4 involved the same ambiguous rule, against a 60% floor.
- 11.3% vs ≥50% — The agent cited explicit rule codes on 11.3% of records at L4 against a predicted 50%, falsifying that prediction. (Partly a measurement-floor question: the taxonomy's lexicon requires explicit rule-code strings and the agent paraphrases policy provisions more than it cites them. The paper's own reading is that the gap is too large to be explained by lexicon conservatism alone.)
- 7 ALLOWs withdrawn — Adding governance prose made the agent less committal, not more. All seven L0 ALLOW records withdrew to REVIEW once the L1 prose framed the substrate as procurement governance.

{/* The PDF remains the version of record. The summary, method, findings
    and limitations above are frontmatter, not body prose — they render
    through the research template's structured blocks (PSA-040) so that
    each one is separately labelled, quotable and checkable against the
    section of the paper it came from. The MDX body stays empty by
    design: this page frames and summarises the publication, it does not
    become it. */}