Short answer first: no. But what MIT actually showed in Beyond Binary Rewards: Training LMs to Reason about Their Uncertainty is more interesting than the headline. Damani, Puri, Slocum, Shenfeld, Choshen, Kim, and Andreas demonstrate that the way we currently train reasoning models doesn't merely allow hallucinations – it actively produces them. And they have a fix for one specific class of the problem.
The pattern every developer knows
Anyone working seriously with Claude, GPT, or DeepSeek-R1 has seen it: the model answers with absolute confidence – and is completely wrong. Andrej Karpathy has described this behavior across multiple posts, and my recent analysis of a CLAUDE.md with 25,000 stars was, at its core, a collection of behavioral rules aimed at exactly this pattern.
The MIT paper now provides the explanation at the training level. The thesis in one sentence: reinforcement learning with binary correctness signals – today's standard for reasoning models like o1, DeepSeek-R1, or the Qwen reasoning series – rewards guessing exactly as much as knowing. Abstaining is penalized identically to being wrong. The consequence: models learn to bluff with confidence.
The authors put it bluntly in the introduction: post-RL reasoning models exhibit "worsened calibration and increased hallucination rates compared to base models." The training step that makes models better at solving hard problems simultaneously makes them less reliable at admitting when they don't know.
Why binary rewards train models to guess
The crux fits in one diagram. Left: the standard setup. Right: the authors' proposal.
Left: under the standard reward, the model's confidence is irrelevant – a confidently wrong and a hesitantly correct output are worth exactly the same. Right: the calibration reward punishes confidently-wrong answers and rewards calibrated confidence. After Damani et al. 2025, Figure 2.
On the left, the reward is a step function: 1 if the answer is right, 0 if not. The confidence variable q doesn't appear in the reward at all. The model has no incentive to be honest about uncertainty – it can only win by guessing. The authors formalize this as the actual training objective: maximize correctness, ignore everything else.
On the right, the reward itself becomes a function of confidence. A correct answer at q=1.0 gets full reward. A correct answer at q=0.3 gets only partial reward – the model was right but didn't know it. A wrong answer at q=0.9 is heavily punished – the model lied with confidence. A wrong answer at q=0.1 ("I'm just guessing") gets only a mild penalty.
The fix: RLCR in one paragraph
The authors call their method RLCR (Reinforcement Learning with Calibration Rewards). The reward function:
R = correctness + (1 − (q − correctness)²)
The trailing term is the Brier score, a rule used for decades in weather forecasting to evaluate calibrated probabilities. Theorem 1 of the paper proves the central property: this reward function maximizes accuracy and calibration simultaneously – with no tradeoff. The model doesn't learn to prefer uncertainty over certainty. It learns to output a confidence that matches its true probability of being correct.
In practice the output looks like this (simplified from the paper, Figure 1a):
<answer> "Boom Bang-a-Bang" </answer>
<analysis> Uncertainty is high: Lulu represented the UK in 1969, but the specific song isn't widely known […] </analysis>
<confidence> 0.3 </confidence>
Four structured fields: reasoning, answer, a self-analysis of the uncertainty, a numerical confidence. The model is trained so that this confidence isn't arbitrary but statistically tracks reality.
The numbers that matter
On HotpotQA – multi-hop questions over Wikipedia – Expected Calibration Error drops from 0.37 to 0.03. Translated: an RLVR-trained model that answers with confidence 0.9 is on average right 53% of the time. An RLCR model at the same confidence almost exactly matches its true success rate. On the math suite (GSM8K, MATH500, Big-Math), ECE falls from 0.26 to 0.10. Accuracy stays essentially unchanged.
The most striking result hides in the out-of-distribution tests. When the trained models are evaluated on new datasets (TriviaQA, SimpleQA, GPQA, CommonsenseQA), something disturbing happens: standard RL doesn't merely fail to improve calibration – it makes it worse than the untrained base model. RLCR is the only method in the comparison that transfers its calibration gains to new tasks.
As a bonus, the authors show verbalized confidence is usable for test-time scaling: confidence-weighted majority vote beats both vanilla majority vote and pure max-confidence selection. If you're already using self-consistency or best-of-N, the same training recipe gives you a better voting signal for free.
So did MIT solve hallucinations? (The honest part)
Three things to say in the same breath:
What it solves
Calibration on tasks with verifiable ground truth – QA, math, structured reasoning. Models trained this way reliably know when they're guessing.
What it doesn't solve
Open-ended factual hallucinations in domains without ground truth. RL needs a correctness signal. "Summarize my codebase" or "write marketing copy about X" aren't verifiable tasks – the method doesn't directly help here.
Scale caveat
Training was on Qwen2.5-7B. Whether the effects hold at frontier scale is genuinely open. The theoretical properties (Theorem 1) hold regardless of model size; the empirical generalization effects don't necessarily.
The deeper point lies underneath: binary reward signals are actively harmful for calibration. That's a finding the entire post-training field needs to sit with – including the labs whose models we run in production today. The research has plugged less of a hole than it has surfaced a systematic flaw in the current training paradigm.
What developers can do today
Months to years will pass before RLCR shows up in frontier models – if it ever does. But the failure pattern the study diagnoses is at least partially mitigatable at the application layer:
- Demand confidence in the prompt. The models aren't perfectly calibrated, but forcing them to emit a confidence value gives downstream logic something to work with. Set a threshold, route low-confidence outputs to validation.
- Route low-confidence outputs in agent systems. In multi-step agents, critical decisions with low self-rated confidence should be passed to verification steps or human review.
- Self-consistency with confidence weighting. If you use best-of-N or majority vote, weight votes by the model-reported confidence. This is the same voting scheme the paper shows beats plain majority vote.
- CLAUDE.md rules against bluffing. The "ask when ambiguous" rule Karpathy proposes in his coding observations is, in retrospect, a behavioral patch for exactly the training-level problem this paper diagnoses on the RL side. Both approaches address the same underlying pathology from different directions.
The actual result
MIT didn't solve AI hallucinations. They showed that the way we train models systematically produces them – and that a small change to the reward signal mathematically provably yields models that are simultaneously more accurate and more honest about their limits.
That's bigger than the headline. A headline promises an end to the problem. This study delivers a different diagnosis: the problem is built into the training itself. Once you understand that, you also know how to spot it as a user – and what to do about it at the prompt and system layer, long before the next frontier model ships the fix in its architecture.
Sources:
Building AI systems where model confidence and hallucination risk matter? Let's talk. I help design agent architectures that treat uncertainty as signal instead of hiding it.