A language model produces the most plausible continuation of the text it has been given. It has no separate store of facts to check against and no mechanism that distinguishes recalling something from constructing something that fits the pattern. Fluent invention is therefore not a bug in the ordinary sense — it is the same process that produces correct answers, applied where the model has nothing reliable to draw on.
Why it sounds so confident
Training rewards helpful, well-formed answers. Hedging and refusal are learned behaviours layered on afterwards, and they compete against a strong pull towards producing something. The result is that a model's tone carries no information about its reliability — the invented citation reads exactly like the real one.
What actually helps
Grounding the answer in retrieved text and requiring citations, so a claim can be checked against a source. Giving the model an acceptable way to decline, and testing that it uses it. Constraining the output to a schema for anything structured. Asking for the answer in a form that can be verified programmatically wherever possible.
What helps less than people think
Telling the model not to hallucinate. Asking it how confident it is — the stated confidence correlates with the tone of the question more than with correctness. Raising the temperature to zero, which makes output deterministic without making it true.
The design conclusion
Build so that a wrong answer is caught rather than so that a wrong answer is impossible. That means citations a reader can follow, a human in the loop where the stakes justify one, and never using unverified generated text as the input to something consequential. Systems designed on that assumption remain useful; systems designed on the assumption of reliability fail in ways their users cannot see.