Method

How a claim gets here.

Every row in the ledger carries a badge saying what was checked. This page says what those checks actually prove — and what they do not.

Where the ledger stands today

• 0 of 9 claims carry a recorded machine check and the quotes behind it.
• 9 of 9 carry no such record.
• 0 of 7 cited papers have been read end to end by a person.

That last number is zero. No paper behind any claim on this site has been read start to finish by a human. Where a claim ships a quote, that quote is verified to appear in the paper — but nobody has yet confirmed that the surrounding argument says what the claim implies. This is published rather than hidden for the same reason stale rows are flagged rather than quietly refreshed: a ledger about decay that conceals its own decay is worth nothing.

The pipeline

1. Search. A wishlist of technique and task pairings drives an arXiv search; some rows pin specific papers by id.
2. Identity check. The fetched document is confirmed to be the paper it claims to be, before any model is paid.
3. Extract. A model drafts a structured claim, quoting the passages it relied on.
4. Verify, deterministically. Every quote must appear verbatim in the fetched text. Every number asserted in prose must appear inside one of those quotes. Neither check involves a model.
5. Editorial rules. The draft must satisfy the same validation the site enforces at build time, or it is discarded rather than costing a review.
6. Human review. Whatever survives opens a pull request that a person reads and merges, rejects, or corrects.

What each badge means

quotes verified · numbers traced

Both deterministic checks in step 4 passed when this claim was drafted. It means the quoted text really is in the paper and the figures really came from those quotes. It does not mean the claim is true. A quote can be genuine and still be misread — taken from an ablation, a different model size, a baseline rather than a treatment.

no check recorded

One of two things, and the ledger cannot currently tell them apart. Either the claim was written by hand before the pipeline existed, or it was drafted by the pipeline and did pass the checks, but before that record was kept — and its quotes cannot be recovered, because the pull request that carried them flattened all of a claim's quotes into one list without recording which finding each belonged to. Rather than guess at that mapping, these rows claim nothing.

not read in full

Nobody has confirmed the claim against the whole paper. Shown per source.

not re-checked since …

Older than nine months. The claim may have been true about models that no longer exist.

What cannot be checked here

The verification in step 4 runs once, when a claim is drafted, and cannot be repeated when the site builds: the papers are fetched into a local cache that is deliberately never committed or deployed. So the badge records that a check ran, not that it would pass again today.

That is why every checked claim ships the quote it relied on. The quote is the part you can test yourself — follow the citation, search the paper, and see whether the passage says what this site says it says. If it does not, the ledger is wrong and we would like to know.

If you think a row is wrong

Every claim carries a Flag an error link that reports that specific row. Scope errors are the most valuable thing you can send: a paper that evaluated one model family does not license a claim about all of them, and that is the failure quote-checking cannot catch.