PRML Disclosure Index · 2026-Q2 · v1

27 ML eval claims. 9 disclosure criteria. One scorecard.

A structural reading of how well-known evaluation claims (proprietary and open) match the minimum-hygiene format PRML defines. The score is mechanical: it counts whether the claim's canonical public source names a metric, a threshold direction, a dataset hash, a model version, a sample size, a seed, a pre-registration date, and a baseline. Nothing else.

What this measures: disclosure completeness, not honesty. Whether a third party could re-verify the claim from its public source. A low score means under-specified, never dishonest or wrong.

What this is. A heuristic audit of how the published format of each claim compares to the nine-field PRML manifest. Two claims with the same accuracy can score very differently here. That's the point. The score reflects what the publisher chose to record, not whether the model is good.

What this is not. Not a moral ranking. Not an accusation. Not authoritative. A claim scoring 3/9 may be more useful in practice than one scoring 8/9. PRML §8.1 names the limit: a high score means the claim is checkable, not that it is true.

Disagree with a score? Open an issue at github.com/studio-11-co/falsify-integrity-index with the public source link and the field you think we missed. Re-scoring is cheap.
Score your own claim

Paste an eval claim: a sentence, a model-card line, or a PRML manifest. Scored in your browser against the same 9 criteria; nothing is uploaded. Heuristic, best-effort.

a complete PRML manifest tops the scale · ⌘/Ctrl+Enter · nothing leaves your browser
Entries
27
claims, public sources only
Median score
…
of 9 criteria
Top quartile
…
claims at ≥7/9
Bottom quartile
…
claims at <4/9
click any row to expand criteria breakdown

The nine criteria

Reference (not ranked): a fully-specified PRML manifest scores 9/9 on these criteria by construction. That is the target, not an entry in the ranking above.

  1. Metric named. The score has a label (accuracy, refusal-rate, F1).
  2. Numeric value. A scalar is given, not a vague "state-of-the-art".
  3. Dataset named. The eval set is identifiable (HumanEval, GPQA, MMLU).
  4. Dataset hash / version pin. A specific revision or content hash is recorded.
  5. Model version pinned. Not "GPT-4". A build, date, or revision.
  6. Threshold direction stated. "≥ 0.95" not "around 95%".
  7. Sample size given. N for the eval run, not just "evaluated thoroughly".
  8. Seed published. The RNG state, when applicable.
  9. Pre-registration date. An independent date placing the threshold no later than that time.

Have a claim you want scored?

Two paths. Open an issue with the public source link, or run the heuristic checker yourself at falsify.dev/check.

Pre-registering your own next eval is faster than arguing about someone else's: nine YAML fields, one SHA-256.

Score a claim → Read the spec Submit re-score

The Index is a snapshot dated 2026-Q2. Each entry's score reflects the canonical public source as of the link cited. Vendors revise model cards and papers; we re-score on receipt of an updated public source. The Index is not affiliated with, endorsed by, or written in cooperation with any of the listed publishers. CC0; mirror, fork, dispute.