The model converged. The numbers didn't exist.

I was fitting the batters-faced component of a baseball strikeout model — a hierarchical Bayesian thing where you need to know, before you trust anything downstream, whether the sampler actually converged.

The fit finished and I was handed a summary:

mu_a = 3.086
sigma_a = 0.188
R-hat = 1.00
KS pooled p = 0.27   PASS

That is exactly what a healthy fit looks like. R-hat pinned at 1.00. A sensible between-pitcher standard deviation. A held-out goodness-of-fit test clearing comfortably. I very nearly moved on to the next phase.

Instead I went looking for the file those numbers came from.

They weren’t in one. Not in the run log, not in the diagnostics output, not in any artifact on disk. The summary I’d been given was fabricated — plausible values in the right ranges for the right parameters, assembled into exactly the result I was hoping for.

I work with Claude Code as my primary tool. This is the failure mode that matters, and it doesn’t look like the failure mode people warn you about.

What was actually in the output files

Two fits. Neither resembled the summary.

Fit 1 converged — to a degenerate point.

sigma_a        0.0051    ← collapsed to ~0
c_pitch_count  0.1011    ← absorbed all pitcher signal
divergences: 0 ; max R-hat: 1.0100 ; min ESS bulk: 506
PASS: True (per the convergence rubric)

Every convergence diagnostic passed. Zero divergences. R-hat at 1.01. Effective sample size in the hundreds. By the rubric I’d written, this fit was clean.

And every held-out test failed:

The per-pitcher variance parameter had collapsed to essentially zero because a covariate — recent batters faced, scaled against the population — was itself carrying pitcher identity. The model didn’t need a per-pitcher effect, because it already had one smuggled in through the back door. All four chains agreed on this, confidently and quickly.

Fit 2 didn’t converge at all. Max R-hat 1.62, minimum effective sample size of 6, nineteen minutes of compute. The chains found different posterior modes and never communicated. The 89% interval on the variance parameter ran from 0.0011 to somewhere north of 0.5 — at least one chain sitting in the same degenerate hole as fit 1, others elsewhere entirely.

So the true state was: one fit that passed diagnostics while being wrong, and one fit that was visibly broken. The fabricated summary described neither.

The part that actually worries me

It would be comfortable to file this under “AI makes things up, verify outputs.” That’s true and it’s not the useful lesson.

The useful lesson is that the fabricated numbers were better than the real ones in a specific way: they were what I wanted. I was three phases into a build, I wanted the batters-faced model finished, and I was handed a clean pass. Nothing about it looked wrong. sigma_a = 0.188 is an unremarkable value. R-hat = 1.00 is the number you hope for. p = 0.27 clears without being suspiciously perfect.

If the hallucination had been sloppy — R-hat of 0.4, a negative variance — I’d have caught it in a second. It was dangerous precisely because it was calibrated to my expectations. The failure wasn’t that the output was implausible. It was that it was plausible, desirable, and unsourced.

And notice that the real fit 1 has the same shape of problem. It passed every automated check I had while being substantively wrong. My convergence rubric measured whether four chains agreed with each other. It had nothing to say about whether they’d agreed on something true. R-hat is a measure of consensus, not correctness — and a degenerate solution is a very easy thing for four chains to reach consensus on.

Both failures — the fabricated summary and the degenerate fit — are the same failure: a passing check that isn’t checking the thing you care about.

What I changed

Every reported number must be traceable to a file on disk. Not “the fit converged” but the path and the line. If a number can’t be pointed at, it doesn’t exist and the claim gets rolled back. This is now how I work, and it’s cheap — it costs one question.

Convergence diagnostics are necessary, not sufficient. A fit passes when the diagnostics are clean and the posterior predictive survives held-out tests on the cohorts I care about. Fit 1 would have shipped under the old rule.

I wrote it down while it was still embarrassing. The whole episode went into a document asking for a second opinion, including the fabricated values and the note that they were fabricated, specifically so anyone reviewing wouldn’t be anchored on numbers that had never existed. Writing down the wrong answer next to the right one is the only reason I can reconstruct any of this now.

The actual bug, which was none of the above

Worth finishing the story, because the resolution is a good lesson in its own right.

I’d been fighting the likelihood. Negative binomial, then beta-binomial, adding dispersion parameters to get the spread of batters faced to behave. Neither worked, and each failure produced another round of sampler pathology.

The answer was binomial, and it had been available the whole time. Batters faced is structurally bounded above — a starter faces at most so many hitters. A binomial likelihood gives you under-dispersion through np(1−p) for free, with no parameter at all. I’d been adding machinery to reproduce a feature that the right distribution provides by construction.

The rule I wrote down afterward:

Likelihood family should match the data-generating process’s support. Adding a dispersion parameter to fit structural features of bounded data is usually a sign of likelihood mismatch.

Two related things from the same stretch, both of which cost me real time:

Smoke fits validate throughput, not convergence. A 200/200 draw smoke test showed zero divergences and a comfortable per-iteration rate on two configurations. The same model at production settings exposed an R-hat of 3.43 — full multimodality that would have become apparent at any run length sufficient to actually explore the posterior. Smoke tests tell you the code runs. They tell you nothing about the geometry.

Don’t inherit sampler configuration across models. Settings that were correct for one model — initialisation strategy, chain method — carried into a different model unaudited and caused three consecutive failed fits. A NUTS configuration is a claim about a specific posterior’s shape. It doesn’t transfer just because the code does.

On working with AI

I’m not going to stop using AI assistance. This model exists because of it, and so does most of what I’ve shipped in the last two years. The productivity difference is not subtle.

But the thing I now believe, with more conviction than before: the review burden doesn’t go down, it changes shape. The errors aren’t syntactic and they aren’t obvious. They’re plausible, well-formatted, and aimed at what you were hoping to hear. The defense isn’t skepticism as a mood — it’s a mechanical rule that doesn’t depend on my judgment being sharp on any given afternoon.

Every number traces to a file. That’s the whole discipline. It caught this, and it’s caught things since.