The finding
A diffusion model's answer hides its doubt.
The journey there keeps it.
At every denoising step the sampler commits the positions it is most sure about and re-masks the rest. The final distribution is therefore sharpened by the sampler itself — it looks equally certain whether the answer is right or wrong. But the step-by-step trajectory still records which positions were contested, and for how long.
Read the two traces left to right. They separate immediately and stay separated — hallucinated answers travel through higher, longer-lived early entropy. Then read the third readout: the endpoint confidence is identical to three decimal places. That single row is the paper's argument. A score computed from the destination cannot tell these two apart; a score computed from the journey can.
Watch a hallucination form
These are real POPE examples: a real photograph, the real question the model was asked, and the real per-step entropy captured at the answer position. Press play and the model answers in front of you. The large token settles as the measured entropy falls; the meter under it is that entropy. Nothing here is simulated.
Start with the pair below. Both end at a final confidence of 0.9999. One is right and one is a hallucination, and the endpoint cannot tell you which.
Question asked—
All 60 captured examples
Every example we captured, not a selected few. Orange marks the ones the model got wrong. Click any of them to run it through the viewer above.
Output confidence is not just weak here. It inverts.
The score almost every abstention method consumes is output confidence. On an autoregressive VLM it works. On LLaDA-V it sits at chance on POPE and below chance on GQA — meaning a confident answer is, if anything, slightly more likely to be wrong.
Where in the trajectory the signal lives
Drag through the steps to see individual captured traces rather than the mean. The answer locks in early; after it locks, the endpoint tells you nothing, while the steps before it already did.
From a score to a guarantee
Ranking errors is useful; a deployment needs a promise. Learn-then-Test turns the score into a threshold that certifies a risk budget. Abstaining by the trajectory answers the most reliable half of POPE at 95% accuracy. Abstaining by confidence leaves 21% error on that same half — worse than the 17% base rate you started with.
Every table in the paper
All fourteen, with the paper's own numbering, parsed directly from the submitted source so the numbers cannot drift apart. The headline AUROCs are recomputed here from the released per-example records — predict-wrong orientation, ties midranked — and reproduce the paper exactly.
Notation and abbreviations
Every symbol and shorthand used in the paper, in one place.
Every figure in the paper
All ten, with their full captions, in the paper's order.
The whole detector
No training, no extra forward passes, no architecture change. One hook on the LM head collects what the decode already computed, and the score is a mean.
# One forward hook on the LM head. The decode is unchanged; we only read it. hook = model.get_output_embeddings().register_forward_hook(grab) def seismo_score(steps): """steps[t] = the answer position's distribution at denoising step t. Returns an error score: higher means more likely wrong.""" H = [-(p * p.clamp_min(1e-9).log()).sum() for p in steps] return sum(H) / len(H) # mean per-step entropy # That is the whole thing. On LLaDA-V / POPE it reaches 0.778 AUROC, # where the model's own output confidence reaches 0.504.
What it does not do
The signal diagnoses; it does not repair. We tried: re-ordering the sampler's commits, voting over steps, and preference training. None recovers the answer, and the reason is measurable — for most hallucinations the correct answer is never the internally favored option at any layer, so a decode-time rule has nothing to select. The advantage is also conditional: it appears under dynamic decoding, and every dynamic cell we have is LLaDA-initialized. On models that commit in roughly one step, plain confidence is already fine.