Scientists at the Oxford Internet Institute are sounding the alarm about a concerning trend in Large Language Models (LLMs) used in chatbots: their ability to hallucinate. These models are designed to generate helpful and convincing responses without any guarantees regarding their accuracy or alignment with fact.

Researchers have published a paper in Nature Human Behaviour, emphasizing that LLMs are not reliable sources of information. While they are often treated as knowledge sources and used to generate information in response to questions or prompts, the data they are trained on may not be factually correct.

One reason for this is that LLMs often rely on online sources which can contain false statements, opinions, and inaccurate information. Users frequently trust LLMs as human-like information sources due to their design as helpful, human-sounding agents. This can lead users to believe that responses are accurate even when they have no basis in fact or present a biased or partial version of the truth.

To combat this issue, researchers urge the scientific community to use LLMs as “zero-shot translators.” This means that users should provide the model with the appropriate data and ask it to transform it into a conclusion or code rather than relying on the model itself as a source of knowledge. By doing so, it becomes easier to verify that the output is factually correct and aligned with the provided input.

While LLMs will undoubtedly assist with scientific workflows, it is crucial for scientists to use them responsibly and maintain clear expectations of how they can contribute. By doing so, we can ensure that these tools help us advance scientific knowledge while avoiding misinformation and bias.