2026-05-22 POST-MORTEM ~5 min

Lock #2: the first thing PRML falsified was its own distribution hypothesis. The second was its own counting bug.

The preregistration and verdict mechanism behaved as designed. The outreach target was missed. Lock #2 resolved at 0 / 3 tonight: zero independent contributors filed RFC engagement during the two-week window. An honest account of what missed, the counting bug we caught two days before, and what changes for v0.3.

Update — 2026-08-23. Three statements in this post claimed more than the result behind them. The body is corrected; the original wording is quoted here so the change is not silent. No number, verdict or registry record was altered: Lock #2 still reads 0 / 3, Lock #1 still reads its corrected count, and both remain on the registry.

1. Point 2 was headed “The diagnosis is a distribution problem, not an interest problem.” A count of zero is consistent with either, and nothing was measured that separates them. The point now states what the result establishes and what it does not.

2. The same point called the RFC text “technically sound” before listing the schemas, vectors and appendix. That is a grade in front of its own evidence; the evidence now stands on its own.

3. Point 4 closed: “If a critic finds the next bug, that’s still PRML doing its job; if we find it first, that’s also PRML doing its job.” A claim that reads every outcome as a win cannot be falsified, which is the opposite of what this specification is for. It now states the boundary instead: PRML made the claim and the correction auditable against each other; it did not catch the bug and does not make counting code correct.

Also since publication: point 3 said the RFC text becomes “v0.2 final” on the scheduled date. The RFC did freeze on 2026-05-22, but it froze without adopting three forward-promises it had carried, and the canonical position today is that v0.1 is the version to implement while the specification itself remains a working draft under public review. See the v0.1 errata.

On 2026-05-08 I locked a public commitment:

Lock #2 — external_contributor_count for the PRML v0.2 RFC, target ≥ 3, resolve date 2026-05-22 23:59 UTC.

Hash: 6d85e0c3d3af446733f8777449f3ff617889dba99a04f7f27c73be763dedf98c.

Today the lock resolved at 0 / 3. Zero independent contributors filed RFC engagement (issues or PRs against rfc-v0.2-labelled tracks on studio-11-co/falsify) during the two-week window.

That's a real miss. Not partial. Not "we were close." Zero.

Four notes

1. The preregistration and verdict mechanism behaved as designed. Lock #2 exists exactly so that a missed commitment can't be quietly re-framed after the fact. The hash was public, the target was public, the resolve date was public. The fail is now public too — automatically, via the same registry, with no admin intervention from me. If Lock #2 had soft-failed silently, the whole pre-registration thesis would be theatre.

2. What the result establishes, and what it does not. It establishes that the engagement target was missed. It does not, on its own, separate insufficient distribution from insufficient interest — a count of zero is consistent with both, and I have no measurement that distinguishes them. What I can state about the artefact is observable rather than evaluative: the JSON Schemas validate, the conformance vectors reproduce byte-equivalently across four reference implementations, and the field-name drift between v0.1 and the v0.2 RFC is documented in the analysis appendix. What I did not do is get the RFC in front of people who are paid to read RFCs in this domain. That is a thing I did not do, which is different from a thing I have measured.

3. v0.2 still freezes today, with founder-only authorship. The version doesn't get postponed because the engagement target was missed — that would be moving the goal-posts. The RFC text becomes v0.2 final on the originally-scheduled date.

4. Two days before this lock resolved, a registry-side audit found the counting routine for Lock #1 was buggy. The unique_producer_count source was supposed to deduplicate producer.id values across all committed manifests, excluding the project's own producer identifiers (e.g. falsify.dev). It was instead treating quoted ("falsify.dev") and unquoted (falsify.dev) values as distinct, and falling back to a regex on truncated previews that captured dataset.id and model.id as if they were producer.id. The displayed count was 8; the correct count is 2 (two test-placeholder producer.id values — example.com and onboarding-walkthrough-test — both also visible on /board, neither a real external user). The bug was fixed and deployed on 2026-05-20. The lock manifest itself was not modified — only the counting code that observes it. The result: Lock #1 is now correctly reading 2/25 with 26 days left, not the inflated 8/25 we'd been showing. This is what running the mechanism on yourself looks like in practice — and it marks a boundary worth stating plainly: PRML made the committed claim and this correction auditable against each other. It did not catch the bug, and it does not make counting or measurement code correct. That is a separate discipline, and here it failed for eleven days.

What changes for v0.3

Things that did happen between lock and resolve

Stating this without celebration. The contributor target slipped; the rest of the work did not.

30-day cool-off in effect from publication. No proactive evangelising for the next 30 days. The public mechanism speaks for itself. Comments are welcome; reframing arguments are not answered.
See Lock #2 on the registry → Read the v0.2 RFC Comment on GitHub
Cüneyt Öztürk — founder, Falsify OÜ; author of PRML. hello@falsify.dev. This post is CC BY 4.0; reuse, fork, cite without attribution if it helps. The Lock #2 hash and resolve metadata are content-addressed on the public registry — anyone can re-derive both from the canonical bytes.