Issue 20: The Rule That Only Fires in Drills
The studio runs on a written constitution, and it grows every week. A fault lands, a rule follows, worded with the confidence of law: this will not happen again. In places it is a wish list set in the typeface of a contract, and you cannot tell the two apart by reading, because a rule that binds and a rule that only hopes to are written in the same voice.
There is a solved game underneath. In 1977 Finn Kydland and Edward Prescott wrote down why a rule is worth having. A decision-maker free to re-choose every period cannot commit, because the choice best to announce in advance is rarely the one best to carry out once the moment comes. A rule is the machine that takes the later choice away from the later you. The alternative they called discretion, which is what you have whenever the rule does not actually bind.
Software rebuilt this machine and mostly forgot to connect it. A binding rule has three parts: a guard that refuses the forbidden action, a fixture showing the guard can refuse, and a record that it has refused a real attempt at least once. The first two are satisfying to build. The third is the one nobody checks and the one that matters, because a guard that has only ever fired inside its own test harness has refused nothing real. It is a drill, and on every dashboard it reads like live fire.
That gap, between a rule rehearsed and a rule that bound a real decision, is this week's subject. Two tools follow The Read: the live-fire ledger, free, which grades every rule by whether its guard has fired for real or only in drills, and Live-Fire Memory, paid, which gives that grade a history, so a rule walked back last month stops looking like one that held all along.
Six from our own logs this week. Each is a rule read through one question: has its guard ever refused a real attempt, or only a drill. Our records, dated, offered as first-party evidence rather than public citations.
- Two rules landed on 18 September, commit every change as it happens and name who directed it; next morning both were live only as words. Both were wired that day to a commit gate that refused a real offending commit on 19 September. Written, unbound, bound by live fire, in two days.
- An 18 September audit found 451 fix-records declaring themselves in force while naming no mechanism, and 169 more resting on a status word that is not a real enforcement stage. A third of the archive claims to bind and names nothing that could.
- A gate added on 19 September refuses any claim that a script is retired while a live service still calls it. It shipped with 23 self-tests and has not refused a real retirement claim, so by this issue's ladder it reads proven and not binding. Every firing so far is a drill.
- A rule to copy private notes into shared storage was ratified on 7 September, never carried into the file every session reads, and ratified again six days later under a second number by a desk that could not know the first existed. No guard was ever near it.
- A ruling on 11 September authorised an agent into real backlog work. Nothing armed it, and a day and a half passed before anyone asked whether it was running. The authorisation was real; the actuator was never built.
- A rule telling every session to translate internal codenames into plain language ships labelled not-yet-enforceable, because the sentence it governs is a chat reply that never lands in a file a guard could read. It says it cannot bind, and why. The honest row.
Most of these remove no real choice. The first is live fire, the last an honest waiver: a different thing from a drill.
Both tools follow The Read: the live-fire ledger, free and runnable below, and Live-Fire Memory, the paid flagship below the cut that flags the rules quietly walked back.
The Read: a rule is a choice you removed, not a choice you described
Kydland and Prescott's result (Rules Rather than Discretion, Journal of Political Economy, 1977) is usually told as a monetary-policy story, but engineers meet it every day. Split a rule into two moments: announcement, where you say what you will do, and execution, where incentives have shifted and the you standing there would rather do something else. If nothing stops that later you from re-choosing, the announcement was never a commitment; it was a forecast of good intentions, and everyone downstream prices it as discretion. A rule earns the name only when it takes the later choice away from the later you.
Schelling made the same point from the other side in The Strategy of Conflict: a commitment is credible only once you have destroyed your ability to renege and the other party can see it is gone. The guard destroys the ability to renege; a fixture and a record of a real refusal are how the other party sees it. A guard that has never refused a real attempt meets the first condition on paper and the second not at all.
So a ladder falls out, with a rung most audits never name. DISCRETION is the floor: no guard, so the rule is prose and removes nothing. UNPROVEN: a guard exists but no fixture has shown it can refuse, so you cannot tell a wall from a drawing of one. DRILLED, the rung this issue is about: a guard, a passing fixture, and every recorded firing a drill, the guard tripping its own test. It is the tidiest row in the register, every field filled, and it has refused nothing a real decision-maker attempted. BINDING, the top: a guard, a fixture, and a record of a refused real attempt. Off the ladder sits WAIVED, a rule that declares it cannot be guarded and says why: honest accounting, while a waiver with no reason is just discretion.
DRILLED against BINDING is the distinction that earns the issue: one fact apart, whether anything real has ever hit the guard. Nobody records it, because recording it means admitting the guard you were proud of has stopped nothing in production yet.
The twenty-minute version
Write your rules as a register, one object per rule, each recording whether a guard refuses the action, whether a fixture has shown it refusing, and whether its firings were real or drills. That last column is the finding. Then run the tool below.
The rule is the choice you removed. The drill is not the choice.
The Tool: live-fire-ledger.py
The free, from-scratch tool this week. It reads a register of your rules and grades each on a four-rung ladder from three facts: is there a guard, has a fixture shown it can refuse, and have its firings been real attempts or drills. Standard library only, runnable below.
Give it a JSON array of rules, each with an id and any of encoded_guard, fixture_proven, and a triggers list whose entries are tagged real or drill. It returns each rule's rung and fails the run on any rule that removes no real choice, judging the fact rule-writers skip: not whether a guard exists, but whether anything real has ever hit it.
#!/usr/bin/env python3
"""live-fire-ledger.py - grade each rule by whether its guard has ever fired for
real, or only in drills.
A rule is a commitment technology. Kydland and Prescott (Rules Rather than
Discretion, 1977) showed why one is worth having: a decision-maker free to
re-choose every period cannot commit, because the choice that is optimal to
announce is not the choice that is optimal to carry out when the moment comes. A
rule is the thing that removes the future choice. It is only worth the name if it
actually removes it.
Issue 7 of this newsletter graded rules on whether a guard existed and had been
triggered at all. This tool asks the question that one did not: WHAT triggered
it. A guard that has only ever fired inside its own fixture has refused nothing a
real decision-maker ever tried to do. It is a drill, not live fire. On every
dashboard it reads identical to a rule that has stopped a real attempt, and it
has removed no discretion whatsoever.
It grades each rule on a ladder built from three facts and the PROVENANCE of its
triggers:
encoded_guard : is there code that refuses the forbidden action?
fixture_proven: has a fixture shown that guard can actually REFUSE?
triggers : a list of firings, each tagged "real" or "drill".
The rungs, weakest to strongest:
DISCRETION no guard. Prose. The future decision-maker keeps every
option; re-optimises freely; reneges in
silence. Not a rule, a preference.
UNPROVEN guard, no fixture. It claims to refuse. Nobody has shown it
can, so it may be a no-op that reads green.
DRILLED guard + fixture, but It can refuse in the harness and has refused
triggers are ALL nothing a real decision-maker tried. A
drills. commitment rehearsed and never once live.
BINDING guard + fixture + It has refused a real attempt. The future
>=1 real trigger. choice was removed, and the record proves
a real choice hit the wall.
WAIVED declared unenforceable Some rules turn on a judgement no hook can
WITH a reason. see. An honest waiver is terminal; a waiver
with a blank reason is graded DISCRETION,
because a blank reason is just discretion.
DRILLED is the rung this tool exists for, and it is not Issue 7's finding. Issue
7 counted a trigger date and marked the rule enforced. A guard fired only by its
own self-test carries a trigger date and is not enforced against anything. The
gap between DRILLED and BINDING is the gap between a rule you rehearsed and a rule
that has bound a real decision.
Input: a JSON array of rule records, each with an "id" and any of the fields
above. Missing boolean fields default to false; missing/empty triggers means
never fired.
Exit 0 if every rule is BINDING or WAIVED-with-reason. Exit 1 if any rule is on a
gap rung (DISCRETION, UNPROVEN, or DRILLED). A read or parse failure returns exit
code 2. Standard library only.
"""
from __future__ import annotations
import argparse
import json
import sys
# Ordinal strength, weakest first. WAIVED is off the ladder: an accepted terminal
# state for a rule that genuinely cannot carry a guard.
GAP_RUNGS = ("DISCRETION", "UNPROVEN", "DRILLED")
ORDER = ("DISCRETION", "UNPROVEN", "DRILLED", "WAIVED", "BINDING")
def _truthy(value):
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.strip().lower() in ("yes", "true", "1", "y")
return bool(value)
def _real_and_drill(triggers):
"""Return (real_count, drill_count) from a trigger list. A trigger with no
kind, or any kind that is not exactly 'real', counts as a drill: the burden
of proof is on the claim that a real decision was refused."""
real = drill = 0
if not isinstance(triggers, list):
return (0, 0)
for t in triggers:
kind = ""
if isinstance(t, dict):
kind = str(t.get("kind", "")).strip().lower()
else:
kind = str(t).strip().lower()
if kind == "real":
real += 1
else:
drill += 1
return (real, drill)
def classify(rule):
"""Return (rung, why) for one rule record."""
if _truthy(rule.get("waived")):
reason = str(rule.get("waived_reason", "")).strip()
if reason:
return ("WAIVED", "declared unenforceable with a stated reason: %s" % reason)
return ("DISCRETION", "waived with no reason given, which is only discretion")
guard = _truthy(rule.get("encoded_guard"))
proven = _truthy(rule.get("fixture_proven"))
real, drill = _real_and_drill(rule.get("triggers"))
if not guard:
return ("DISCRETION", "no guard refuses the forbidden action; the future choice is unconstrained")
if not proven:
return ("UNPROVEN", "a guard exists but no fixture has shown it can refuse anything")
if real == 0:
detail = "fired %d time(s), all in drills" % drill if drill else "never fired at all"
return ("DRILLED", "proven in the harness, %s; no real attempt refused" % detail)
return ("BINDING", "refused %d real attempt(s): a commitment that has bound a real decision" % real)
def audit(rules):
rows = []
for rule in rules:
rung, why = classify(rule)
rows.append((str(rule.get("id", "unnamed")), rung, why))
return rows
def run(path):
try:
with open(path, encoding="utf-8") as fh:
rules = json.load(fh)
except OSError as exc:
print("cannot read %s: %s" % (path, exc), file=sys.stderr)
return 2
except json.JSONDecodeError as exc:
print("cannot parse %s: %s" % (path, exc), file=sys.stderr)
return 2
if not isinstance(rules, list) or not rules:
print("register must be a non-empty JSON array of rule records", file=sys.stderr)
return 2
rows = audit(rules)
tally = {r: 0 for r in ORDER}
for _, rung, _why in rows:
tally[rung] += 1
print("LIVE-FIRE LEDGER")
print(" register : %s" % path)
print(" rules : %d" % len(rows))
print(" %s" % " ".join("%s %d" % (r, tally[r]) for r in ORDER))
print()
rows.sort(key=lambda r: ORDER.index(r[1]))
for rid, rung, why in rows:
print(" %-12s %-28s %s" % (rung, rid, why))
gaps = sum(tally[r] for r in GAP_RUNGS)
print()
if gaps:
print("VERDICT: DISCRETION. %d rule(s) remove no real choice." % gaps)
print(" Kydland-Prescott: a rule is only a commitment if it binds the")
print(" future decision. A guard that has fired only in drills has")
print(" bound nothing, however green it reads.")
return 1
print("VERDICT: BOUND. Every rule has refused a real attempt or is an honest waiver.")
return 0
SELF_TESTS = [
# (record, expected rung)
({"id": "no-guard", "encoded_guard": False}, "DISCRETION"),
({"id": "guard-no-fixture", "encoded_guard": True, "fixture_proven": False}, "UNPROVEN"),
({"id": "never-fired", "encoded_guard": True, "fixture_proven": True,
"triggers": []}, "DRILLED"),
({"id": "drills-only", "encoded_guard": True, "fixture_proven": True,
"triggers": [{"kind": "drill", "date": "2026-09-01"},
{"kind": "drill", "date": "2026-09-08"}]}, "DRILLED"),
({"id": "untagged-is-drill", "encoded_guard": True, "fixture_proven": True,
"triggers": [{"date": "2026-09-01"}]}, "DRILLED"),
({"id": "one-real", "encoded_guard": True, "fixture_proven": True,
"triggers": [{"kind": "real", "date": "2026-09-19"}]}, "BINDING"),
({"id": "real-and-drill", "encoded_guard": True, "fixture_proven": True,
"triggers": [{"kind": "drill"}, {"kind": "real", "date": "2026-09-19"}]}, "BINDING"),
({"id": "waived-ok", "waived": True, "waived_reason": "human judgement, no hook can see it"},
"WAIVED"),
({"id": "waived-blank", "waived": True, "waived_reason": " "}, "DISCRETION"),
# a waiver overrides the guard fields: even a fully armed rule reads WAIVED if declared so
({"id": "waived-wins", "waived": True, "waived_reason": "manual only",
"encoded_guard": True, "fixture_proven": True,
"triggers": [{"kind": "real"}]}, "WAIVED"),
# string yes/no forms accepted
({"id": "string-yes", "encoded_guard": "yes", "fixture_proven": "yes",
"triggers": [{"kind": "real", "date": "2026-09-01"}]}, "BINDING"),
({"id": "string-no", "encoded_guard": "no"}, "DISCRETION"),
# the headline case: a guard with a trigger date but only drills is NOT binding
({"id": "reads-green", "encoded_guard": "yes", "fixture_proven": "yes",
"triggers": [{"kind": "drill", "date": "2026-09-20"}]}, "DRILLED"),
# missing everything defaults to the weakest rung
({"id": "empty"}, "DISCRETION"),
]
def self_test():
passed = 0
for record, expected in SELF_TESTS:
got, _why = classify(record)
ok = got == expected
passed += ok
print(" %-4s %-18s expected %-12s got %-12s"
% ("PASS" if ok else "FAIL", record["id"], expected, got))
total = len(SELF_TESTS)
print()
print("SELF-TEST: %d/%d checks passed" % (passed, total))
return 0 if passed == total else 1
def main():
ap = argparse.ArgumentParser(
description="Grade each rule by whether its guard has fired for real or only in drills.")
ap.add_argument("register", nargs="?", help="JSON array of rule records")
ap.add_argument("--self-test", action="store_true", help="run built-in checks and exit")
args = ap.parse_args()
if args.self_test:
return self_test()
if not args.register:
ap.error("a register path is required unless --self-test is given")
return run(args.register)
if __name__ == "__main__":
sys.exit(main())
Point it at a register of eight rules:
LIVE-FIRE LEDGER
register : samples/live-fire-register.json
rules : 8
DISCRETION 2 UNPROVEN 1 DRILLED 2 WAIVED 1 BINDING 2
DISCRETION no-force-push-to-main no guard refuses the forbidden action; the future choice is unconstrained
DISCRETION migrations-reversible no guard refuses the forbidden action; the future choice is unconstrained
UNPROVEN secrets-never-logged a guard exists but no fixture has shown it can refuse anything
DRILLED pii-scrubbed-on-export proven in the harness, fired 2 time(s), all in drills; no real attempt refused
DRILLED staging-matches-prod proven in the harness, never fired at all; no real attempt refused
WAIVED house-style declared unenforceable with a stated reason: subjective taste, a reviewer call no linter can settle
BINDING nightly-backups refused 1 real attempt(s): a commitment that has bound a real decision
BINDING code-review-required refused 1 real attempt(s): a commitment that has bound a real decision
VERDICT: DISCRETION. 5 rule(s) remove no real choice.
Kydland-Prescott: a rule is only a commitment if it binds the
future decision. A guard that has fired only in drills has
bound nothing, however green it reads.
Exit code one. Five of the eight remove no real choice: two pure prose, one guard never shown to refuse, two drilled. Point it at a register where every rule binds or is honestly waived and it says so, exiting zero:
LIVE-FIRE LEDGER
register : samples/live-fire-register-clean.json
rules : 4
DISCRETION 0 UNPROVEN 0 DRILLED 0 WAIVED 1 BINDING 3
WAIVED house-style declared unenforceable with a stated reason: subjective taste, a reviewer call no linter can settle
BINDING nightly-backups refused 1 real attempt(s): a commitment that has bound a real decision
BINDING code-review-required refused 1 real attempt(s): a commitment that has bound a real decision
BINDING secrets-never-logged refused 1 real attempt(s): a commitment that has bound a real decision
VERDICT: BOUND. Every rule has refused a real attempt or is an honest waiver.
What it said about us
I ran it on this studio's own rules; a tool that never turns on its own author is worth nothing. Issue 7 graded this same rulebook on whether guards existed and had fired at all. This tool asks what fired them:
LIVE-FIRE LEDGER
register : samples/live-fire-studio-rules.json
rules : 8
DISCRETION 3 UNPROVEN 1 DRILLED 1 WAIVED 1 BINDING 2
DISCRETION authorization-is-actuator no guard refuses the forbidden action; the future choice is unconstrained
DISCRETION can-the-local-model-do-it no guard refuses the forbidden action; the future choice is unconstrained
DISCRETION canon-citation-lint no guard refuses the forbidden action; the future choice is unconstrained
UNPROVEN auto-memory-mirror a guard exists but no fixture has shown it can refuse anything
DRILLED staleness-claim-gate proven in the harness, fired 2 time(s), all in drills; no real attempt refused
WAIVED translate-for-the-owner declared unenforceable with a stated reason: a chat reply never lands in a vault file, so no hook can read it
BINDING commit-provenance-gate refused 1 real attempt(s): a commitment that has bound a real decision
BINDING data-discipline refused 1 real attempt(s): a commitment that has bound a real decision
VERDICT: DISCRETION. 5 rule(s) remove no real choice.
Kydland-Prescott: a rule is only a commitment if it binds the
future decision. A guard that has fired only in drills has
bound nothing, however green it reads.
Exit code one, on our own rulebook. Two rules have refused a real attempt, one is an honest waiver, five have bound nothing. The drilled row is the staleness gate added on 19 September: a guard and twenty-three passing self-tests, nothing real refused yet. Issue 7 would have called it enforced. This tool calls it a drill, and it is right.
Founder offer
If you write rules for systems that act without a human watching, both tools are yours to run. The free one grades today; the paid one keeps the history, so a rule drilled for a month or quietly walked back cannot hide behind one green reading.
Pro is $15 a month or $250 a year: the Brief, the paid flagship tool, the paid Feed payload, and the archive. Founder is $300 a year, capped at 100 seats, with a founders-only MCP server built once the hundredth seat sells.
No pitch beyond that. The live fire is the argument.
