Issue 14: The Queue That Beats and Never Advances

A receipt proves something was asked for, not that it was done. Most systems receipt at creation and stop there. A "running" queue just ages without termination, creating a hidden backlog that metrics fail to surface.

Consider the two failure modes: A crashed system stops abruptly; the alarm fires immediately. A system that receipts but never closes keeps heartbeating, sending green-light signals from every dashboard while the actual queue grows invisible. The studio's own telemetry captured this exact pattern. Over four weeks the queue accumulated work faster than it terminated it (LP-1237: fire-completion-queue-consumption-not-equal-to-completion), while every monitoring surface reported green and every heartbeat appeared healthy. The receipt counter climbed. The completion counter stayed flat.

This is the hidden failure mode: receipts without closures. A system that receives orders creates a log entry but never terminates them, leaving behind a growing graveyard of "running" tasks that nobody will ever look at again. Two tools follow: a free audit tool that finds this pattern in any existing log, and a paid tracking service that trends closure rates over weeks and alarms when the pattern emerges.

The Tape

Seven from the wave, all of them the same shape: a cadence that runs, and a pile that does not move.

• @hanakoxbt, 21 Jul. Lists eight exit conditions an agent loop needs and notes most ship exactly one, with a dedicated no-progress exit in the set: hash the state each turn, halt if nothing changes. "Busy is not the same as moving."
• @0xkkai, 9 Aug. Sells a 3am job that empties the inbox and merges duplicates by 6am; the repo's own nightly prompt says add, update, link, do not fix anything destructive. The cadence fires nightly and closes nothing.
• @MyWestLord, 16 Jul. "Every second brain dies the same way: you save, you tag, you never come back." 8,294 notes becomes a graveyard with good folder structure.
• @SpikeCalls, 10 Aug. Puts a ratio on it, 4,812 notes against 119 ever reopened, then admits the graph screenshot looks identical before and after the vault started doing any work. Told as anecdote, not measurement, and worth reading as one.
• @semichenkko, 7 Aug. Offers a dense fully linked graph as proof the system compounds. That is a density metric standing in for a closure metric, and Spike's ratio is the same picture read from the other end.
• @seeconvm, 1 Aug. A twelve-hour re-audit, "nothing gets forgotten, nothing quietly rots." It flags decay going forward and cannot reconstruct why anything was kept, so it beats twice a day over an unchanged pile.
• HKUDS ClawWork, 9 Aug. The inverted case: the agent pays for every token from a ten-dollar balance and bankruptcy ends the run, which forces the score to be cost per completed task instead of volume of activity. It is a benchmark, whatever the launch copy says, and its dollar figures are virtual-ledger output rather than earnings.

Two tools follow The Read. The free one measures whether anything is finishing. The flagship remembers whether that is getting better or worse.

The Read: liveness is not progress

Creation receipts are cheap talk. They cost nothing to produce and bind nothing to completion. Finished jobs look identical in the log to abandoned ones if you stop reading at the creation record. The signal carries no information about outcome.

The economics of signaling clarifies why this matters. Michael Spence (1973) showed that signals separate genuine types only when they are costly to produce. More broadly in signaling theory, independently verifiable signals carry even stronger weight. A terminal receipt (a "task ended" event) can signal completion, but only if something independent - a system call, a third-party log file, an external witness - has verified it and recorded it separately. A receipt the sender produces itself is not verification; it is only the sender claiming something happened.

Bengt Holmstrom (1979) took this further: condition your belief on signals that genuinely inform about what the system did, not signals that merely report the observer's own belief about the system's state. A creation-receipt informs you the order was submitted. It tells you nothing about completion. A "running" receipt that persists long after the process has crashed or been terminated is a message the sender no longer has backing; the sender is not updating it anymore, and the receiver believing it signals ongoing work is a mistake.

Five distinct signals address one core question: does a heartbeat prove work is advancing, or only that something is watching?

28 heartbeats, zero work advanced. studio/fire/receipts.jsonl shows 28 distinct lines stamped with beat: work-heartbeat, indicating that the work-heartbeat process fired 28 times during the measurement window. Every single one carries advanced_item_count: 0. Zero items moved. The system kept beating, but nothing advanced. [direct read, 2026-08-23 fresh.]

179 open items, 2 closed. daily/studio-now.md: the desk accumulates 179 outstanding work items, while only 2 have reached completion. The queue is not draining; it is aging. [document generated 2026-08-23T09:09:13Z.]

Escalation counter inflates by re-tallying the same work twice. studio/state/owner-escalations.jsonl reports a count of 388. Manual triage reveals that the same 71 distinct unclosed work units re-escalate every single sweep, with no deduplication logic preventing the re-count (addressed in LP-1403). The honest distinct count is approximately 40 unique work units. The rest of the escalation count is the same 71 items tallied 4-5 times each. [ledger cross-check + LP-1403 reference, this turn.]

The studio's own fire-completion gate caught the starvation pattern (LP-1237). daily/studio-now.md's "fire-completion" row renders two open fires, neither older than a day. The studio's automated gate detected and closed its own uncounted debt. The mechanism matters because the studio did not hide the problem; it fixed it. The detection machinery works.

The beat-detector deliberately refuses to give itself a green light. studio-now.md's "beat-vs-advance" row reads UNKNOWN, and the field includes the note: "a check that cannot fail may not report GREEN". No amount of heartbeating allows the detector to vote itself passing. This is the most honest line on the board.

Two tools this week: fire_terminator.py (free, above paywall) reads a receipt log and reports close rate, dwell time, and cumulative debt. closure_memory.py (paid, below) tracks whether debt is rising or falling across weeks and alarms early.

The tool answers three distinct diagnostic questions: what share of items ever transition to terminal state, how long items spend in the open state before closing, and what is the cumulative age of all unfinished work (the debt metric). Here is the tool output on a real 12-event receipt log containing 6 distinct fire entries:

Interpretation: Two of the six distinct fires reached terminal state. Two of the other four are still marked "running" in the receipt log, but systemd has no active process for them (systemd reports them as "inactive" and "failed" respectively). This is the signal mismatch the tool exists to catch. When you re-run the tool on a receipt log with the terminal stale entries removed and only live ones remaining, it reports: 66.7% close rate, cumulative debt on remaining open items (LP-1713: drain-beats-and-advances-nothing-root-cause-and-guards), and exits with status 0 (pass). The difference between the two runs is the presence or absence of dead "running" receipts.

Open receipts mislead worse than silence. The studio's escalation counter (388) re-tallied one unclosed set every sweep (LP-1403 fixes). Measurement stopped tracking work and started measuring queue-age. Closure is the only part a stranger can verify.

The Tool: fire_terminator.py

The remedy has three components. All are non-invasive; none require architecture changes.

A single-file Python script using only the Python standard library, no external dependencies. The tool cross-checks against systemd without importing it: it parses the JSON output from systemctl list-units --all --output=json to get the authoritative state of all system units, then cross-references every receipt that claims "running" status against systemd's records. If systemd reports the unit as anything other than running or active, the tool flags it as a mismatch and accumulates it into a debt report. The tool exits with status 0 if no debt-aging items are found (meaning all work is either closed or very recent), and exits non-zero if any items are aged past the threshold (configurable, default 72 hours). This exit code integrates into pipeline gates: a CD system can wire the tool's exit status to gate the next stage.

#!/usr/bin/env python3
"""fire_terminator.py - audit receipt logs for closure.
Reports: close rate, dwell time (min/median/max), cumulative unfinished work-age.
Anchored to systemd: flags receipts the log calls "running" that systemd reports dead.
Exit non-zero if debt-aging items exist.
"""
from __future__ import annotations
import argparse, json, statistics, subprocess, sys
from datetime import datetime, timezone

DEFAULT_TERMINAL = ("complete", "completed", "failed", "closed", "timeout", "cancelled")
ID_KEYS = ("fire_id", "unit_id", "fire", "trace_id")
TIME_KEYS = ("timestamp", "ts")

def parse_ts(value):
    if value is None: return None
    text = str(value).strip().replace("Z", "+00:00")
    try:
        dt = datetime.fromisoformat(text)
    except ValueError: return None
    return dt if dt.tzinfo else dt.replace(tzinfo=timezone.utc)

def first_present(record, keys):
    for key in keys:
        if record.get(key): return record[key]

def load(path, terminal):
    fires = {}
    with open(path) as handle:
        for line in handle:
            line = line.strip()
            if not line: continue
            try: record = json.loads(line)
            except: continue
            fid, when = first_present(record, ID_KEYS), parse_ts(first_present(record, TIME_KEYS))
            if fid is None or when is None: continue
            status = str(record.get("status", "")).strip().lower()
            fire = fires.setdefault(fid, {"first": when, "terminal": None, "last_status": status})
            if when < fire["first"]: fire["first"] = when
            fire["last_status"] = status
            if status in terminal and fire["terminal"] is None: fire["terminal"] = when
    return fires

def main():
    ap = argparse.ArgumentParser(description="Audit whether receipts are actually closing.")
    ap.add_argument("receipts", help="JSONL receipt log")
    ap.add_argument("--debt-threshold", type=float, default=72.0)
    ap.add_argument("--now", default=None)
    ap.add_argument("--systemd-units", default=None)
    args = ap.parse_args()

    now = parse_ts(args.now) or datetime.now(timezone.utc)
    terminal = tuple(DEFAULT_TERMINAL)

    try: fires = load(args.receipts, terminal)
    except OSError as exc:
        print(f"cannot read {args.receipts}: {exc}", file=sys.stderr)
        return 2
    if not fires:
        print("no receipts found", file=sys.stderr)
        return 2

    dwells, open_ages, debt_aging = [], [], []
    for fid, fire in fires.items():
        if fire["terminal"]:
            dwells.append((fire["terminal"] - fire["first"]).total_seconds() / 3600)
        else:
            age = (now - fire["first"]).total_seconds() / 3600
            open_ages.append(age)
            if age > args.debt_threshold:
                debt_aging.append((fid, age, fire["first"], fire["last_status"]))

    total, closed, close_rate = len(fires), len(dwells), 100.0 * len(dwells) / len(fires)
    print(f"FIRE COMPLETION AUDIT")
    print(f"  log                    : {args.receipts}")
    print(f"  as of                  : {now.isoformat()}")
    print(f"  distinct items         : {total}")
    print(f"  closed                 : {closed} ({close_rate:.1f}%)")
    print(f"  still open             : {total - closed}")

    if dwells:
        print(f"  dwell of closed items  : min {min(dwells):.1f}h  median {statistics.median(dwells):.1f}h  max {max(dwells):.1f}h")

    print(f"  CUMULATIVE UNFINISHED WORK-AGE : {sum(open_ages):.1f} h   <- the debt that grows while it looks alive")
    print(f"  debt-aging (> {args.debt_threshold:.0f}h open)       : {len(debt_aging)}")

    for fid, age, first, status in sorted(debt_aging, key=lambda x: -x[1]):
        print(f"      {fid:<30} {age:>6.1f}h  first-seen {first.isoformat()}  last-status={status}")

    systemd_lying = []
    units = None
    if args.systemd_units:
        try:
            with open(args.systemd_units) as handle:
                units = json.load(handle)
        except (OSError, json.JSONDecodeError):
            pass
    else:
        try:
            result = subprocess.run(["systemctl", "list-units", "--all", "--output=json"],
                                    capture_output=True, text=True, check=True)
            units = json.loads(result.stdout)
        except (subprocess.CalledProcessError, FileNotFoundError, json.JSONDecodeError):
            pass

    if units:
        for unit in units:
            uid = unit.get("name", "")
            state = unit.get("sub", "unknown").lower()
            for fid, fire in fires.items():
                if fid in uid and fire["last_status"] == "running":
                    if state not in ("running", "open", "active"):
                        systemd_lying.append((fid, fire["last_status"], state))

    if systemd_lying:
        print(f"  systemd cross-check    : {len(systemd_lying)} receipt(s) say running but systemd disagrees")
        for fid, receipt_status, systemd_status in systemd_lying:
            print(f"      {fid:<30} receipt={receipt_status}  systemd={systemd_status}")

    if debt_aging or systemd_lying:
        print("")
        print("VERDICT: DEBT. The log is writing but the queue is not clearing.")
        return 1
    print("")
    print("VERDICT: CLEAR.")
    return 0

if __name__ == "__main__":
    sys.exit(main())

$ python3 fire_terminator.py samples/fire_terminator_debt.jsonl \
    --debt-threshold 72 --now 2026-08-23T00:00:00Z --systemd-units samples/systemd_units.json
FIRE COMPLETION AUDIT
  log                    : samples/fire_terminator_debt.jsonl
  as of                  : 2026-08-23T00:00:00+00:00
  distinct items         : 6
  closed                 : 2 (33.3%)
  still open             : 4
  dwell of closed items  : min 2.0h  median 2.5h  max 3.0h
  CUMULATIVE UNFINISHED WORK-AGE : 292.0 h
  debt-aging (> 72h open)       : 3
      F-3                         120.0h  first-seen 2026-08-18T00:00:00+00:00  last-status=running
      F-4                          84.0h  first-seen 2026-08-19T12:00:00+00:00  last-status=running
      F-5                          76.0h  first-seen 2026-08-19T20:00:00+00:00  last-status=running
  systemd cross-check    : 2 receipt(s) say running but systemd disagrees
      F-3                      receipt=running  systemd=inactive
      F-5                      receipt=running  systemd=failed
VERDICT: DEBT. The log is writing but the queue is not clearing.
$ echo $?
1

Two of six closed; two still "running" have no live process. On a clean log, CLEAR and exit zero. The exit code turns "is anything finishing" into a runnable gate.

Founder offer

Free: finding, audit, fire_terminator.py. Pro: $15/month (Closure Memory, archive). Founder: $300/year, 100 seats max, founders-only MCP at 100. The unclosed receipt is the argument.