THE

ANNEX

updated

← AI Today
AI Today · daily note
No.
128
•
•

The Delusional Letter

December 10th, 2025 | By Jorge Rodriguez

U.S. attorneys general warned major technology companies about AI systems that produce delusional or dangerously persuasive outputs, Reuters reported on December 10th. The warning followed growing concern that chatbots can reinforce false beliefs, encourage risky behavior, or respond with confident language in moments of psychological fragility. A state legal officer does not need to prove that a model has intention. The letter asks companies to answer for design, marketing, safety testing, escalation rules, and the commercial decision to place conversational systems in front of ordinary users.

The word delusional is doing awkward work here. It belongs first to human suffering, then gets borrowed to describe software that has no belief, no fear, no private conviction. Yet the borrowing is useful because the harm happens through language that sounds settled. A user receives an answer with the cadence of certainty, often inside a private interface that invites continuation. The legal system is trying to catch up to a product whose danger may appear as a sentence, a refusal that arrives too late, or a recommendation phrased with calm. The companies can adjust filters and warning labels. The state letter turns those adjustments into possible evidence.

No.
128
•
•

Latest from AI Today