ChatGPT is Bullshit — Recent Study claims

Why AI “Hallucinations” Are Better Understood as Bullshit

It’s been over a year and half since Gen-AI and ChatGPT came into limelight. Initially, it seemed that Gen-AI was the future and many people even replaced popular websites like Google or Stackoverflow with ChatGPT and other apps similar to it. However, as these models were adopted by wider audience, issues like AI Hallucinations, where a LLM model would generate false outputs are becoming more apparent.

A recent study from the University of Glasgow takes a stab at AI Hallucinations and this article will summarize their findings. This provocative study argues that ChatGPT’s false outputs should be characterized as “bullshit” rather than “hallucinations”. The authors went further, and claimed that the characterizations should be extended to other neural network based AI-chatbots e.g. Google’s Bard chatbot, AnthropicAI’s Claude (claude.ai), and Meta’s LLaMa.

When The New York Times (NYT) asked a number of AI chatbots when the paper first reported on AI, each answered with a different hallucination that merged fact with invention. In addition, each cited purported NYT articles that did not exist. The reporters spotted the errors easily because they knew the real answer.

Understanding AI Hallucinations:

“What are AI hallucications ?” AI hallucinations occur when an AI model generates incorrect or fabricated information. Unlike humans, who create errors due to misunderstandings or lack of knowledge, AI hallucinations result from the model’s statistical methods of predicting and generating text. These models do not have a true understanding of the world but instead use patterns in the data they were trained on. Large language models like ChatGPT are not even trying to perceive or represent the truth or reality, hence, they are able to produce convincing but false information.

Authors explain, “ChatGPT is not designed to produce true utterances; rather, it is designed to produce text which is indistinguishable from the text produced by humans. It is aimed at being convincing rather than accurate.”

Authors argue that ChatGPT illustrates the philosophical concept of “bullshit” explored by Harry Frankfurt. Lying requires knowing the truth and then having intention to mislead. Whereas, bullshitting involves a complete disregard and ignorance towards the truth.

Soft Bullshit vs Hard Bullshit

In the context of AI, there are two types of bullshit: soft and hard.

Soft bullshit occurs when the speaker or writer is indifferent to the truth. Authors argue that ChatGPT, by design, is not intending to say the truth but to generate text that flow naturally and seems credible. The factual accuracy doesn’t matter.

Hard bullshit involves an active attempt to deceive the audience. Authors insist that ChatGPT is “hard bullshitter”, as it’s intention is to make users believe that the output is based on truth.

Implications of hallucinations:

The study highlights several implications of AI hallucinations for users, developers, and businesses:

Implications for Users:

• Users need to understand that the Ai chatbots have their limitations and they should take the ouputs with a grain of salt.
• Users need to be cautious about blindly trusting the information from ChatGPT and similar models.
• Users are now encouraged by the developers now to do a manual fact-check.

Implications for Developers

• Devs should invest heavily on testing their models for accuracy and update their LLMs with truth tracking features.
• Devs should create feedback mechanisms via which they are able to get the inaccurate data generated from the users and then use that data to further develop their models.
• Dev should not only focus on creating large blocks of content but also on the quality of content created.

Implications for Businesses

• Businesses can suffer financial, legal and reputational impacts due to false data generated by chatgpt. need to be careful of below key points. For example, There has been a case of false legal information being generated due to ChatGPT.
• So far AI bots seems to be doing good as support systems to make existing workforce more productive rather than replacing them entirely.

Mitigation Strategies:

Below are some mitigations strategies mentioned in the study:

• Implementing robust verification process to cross check AI outputs.
• Developing models that can offer greater transparency in their decision making processes. This will be helpful for users to understand how data is generated.
• Continuous monitoring and training of models in supervised enviornment can reduce liklihood of hallucinations.

Conclusion:

The study’s arguments for presenting AI hallucinations as “bullshit” to represent the current limitations of LLMs is something I find really interesting. The authors did a great job to explain their thought process. AI hallucinations create significant challenges in terms of accuracy and reliability. Business sectors sensitive to data accuracy needs to adopt AI with care, e.g. Finance. Acknowledging the current limitations is crucial to manage expectations and move towards the ethical development of AI systems. Hopefully, by implementing the robust mitigation strategies will reduce the pontial risks while expanding the adoption of AI.

Reference:

• The Original Paper by Michael Townsen Hicks, James Humphries and Joe Slater: https://www.researchgate.net/publication/381278855_ChatGPT_is_bullshit