The exam nobody studied for

DELPHI wants laws that hold across organisms, not laws fitted to the organisms they were built on. So before anything was built, one whole clade was set aside. Builders work on the rest. When building stops, an assessor who built nothing re-scores every supported law on the held-out organisms with the same pre-registered pass and fail criteria.

What is held out

The exam roster

Species-rank held-out organisms. Genome-ready means the best NCBI assembly is chromosome-level or a complete genome; NCBI's reference flag alone also marks scaffold and contig assemblies, so it is not used. Trait columns say which public tables cover the species; the values themselves are not shown here and must not be used while building.

Higher taxa also held out (families, orders, genera, the class)

Excluded from both sides

Claims whose organisms are birds

These claims, from the books, name held-out organisms. A builder may not use them as evidence; an assessor can use them as exam questions.

Exam results so far

Each completed claim test is re-run by this thread (which built none of them) with its bird rows as the evaluation set only, and the builder's own pre-registered rule is applied to birds alone, including the builder's minimum sample size. Protocol: holdout/EXAM.md.

Which clade gives the exam enough power?

The hub asked for the clade choice to be settled with data. Each completed claim test was re-run on a scratch copy with birds held out and then with mammals held out, and the builder's own rule, including its minimum sample size, was applied to the held-out species alone. The frozen Aves split did not move; the Mammalia column is a what-if. The finding, stated on its own for the next team: POWER.md (generated from the same files).

Nanopublication records

Every finding of this thread, the inconclusive ones first, is written as a Project Delphi nanopublication: the claim in one sentence, its status in words, the evidence item with the frozen exam's checksum and the numbers as structured counts, the pre-registration, and what it does not establish. They are built from files by holdout/nanopubs/build-nanopubs.mjs, checked with np check and the Delphi lint, and stay unsigned drafts until the key owner decides; the test registry comes first, production only with explicit approval.

Rules for builders

Who has touched a held-out organism? (leak check)

At each publish, holdout/check-leaks.mjs scans every top-level folder except the exam itself and its inputs (listed below with the reason), including every thread's nanopub drafts and Lean modules, for the scientific names, slugs and identifiers of held-out taxa and for the ids of claims whose inventory organisms are all held out. A hit is a question, not a verdict: the file may filter the clade out or be exam material. Every tolerated hit is an allowlist entry with a reason, shown below.

Rulings (questions other threads raised; the allowlist cites them)

Is the roster still frozen?

Anyone can check from a clean checkout:

node holdout/make-split.mjs --verify

Thread status

Status file: status.json. Data: data.json, built from holdout/split.json, holdout/exam.json and holdout/CHECKSUMS.json. Back to Mission Control.