Context – Within weeks of Chat-GPT’s public release, the fact that chatbots make plausible and realistic sounding stuff up emerged. The AI scientists knew about the phenomenon, which they called “hallucinations”. It appears to be baked into the technology. LLMs don’t copy, store and retrieve data. Instead, they are trained on incomprehensibly large data sets to predict sequences of content and can produce remarkable content, especially conversational texts. But they all make stuff up, and there is an increasingly steady stream of media reports of AI hallucinations being found in a wide range of professional documents. Few get included in PEI reports, but I wanted to offer a Holiday Friday count-down of hallucination stories.
(3) An Increasingly Standard Case from the Legal World
Report from the New York Times
A partner at Sullivan & Cromwell, and elite Wall Street law firm, wrote to a federal judge in US Bankruptcy Court in Manhattan to apologize for AI-generated errors in a recent motion. The firm provided a ledger of the errors, which spanned three pages and totaled around three dozen.
Context – Court filings with false cases and false quotes are increasingly numerous. When judges use AI and their rulings include hallucinations it is more noteworthy. Made-up cases making their way into future case documents as precedents is an obvious problem.
(2) Ironic AI Hallucination Runner-up – National AI Policy Written with AI
Report from Techdirt
The South African Government’s proposed national AI policy, which included policy about the dangers of AI-generated misinformation, was clearly written with the help of AI chatbots and contained hallucinated citations. The Government evenutally withdrew the proposal.
Context – Techdirt’s Mike Masnick reported on this unfortunate incident and, as if often the case on digital policy issues, he offers sage thoughts, including that the problem with hallucinations is not that the tech in malicious, but that users of the technology are too often “lazy and uniformed” about the technology’s limitations.
(1) Most Ironic AI Hallucination Story – Book on Truth and AI Includes Hallucinations
Report from the New York Times
Author Steven Rosenbaum’s new nonfiction book titled, The Future of Truth, about the effects of AI on truth, was found to contain multiple fabricated or misattributed quotes during the New York Times’ book review process. He said he accepted responsibility, had launched an internal review, and said future editions would correct the inaccurate passages.
A Final Thought – AI tools are like interns. Great interns. They can be very capable and smart. But the boss, their manager, has to review their work. The boss is responsible. Please check their work!