Issues with AI: Hallucinations
Wise old trees and eager up and comers. That was an answer a large language model or “LLM” provided my colleague. She had asked what was the difference between NCUA historical and current legal opinion letters (LOLs). The AI responded that historical LOLs were the wise old trees of the forest and current LOLs were eager up and comers ready to prove themselves. Suffice it to say, the AI was wrong. It had “hallucinated” the answer, it made it up.
As AI advances and plays a larger role in financial services, credit unions find themselves using AI more and more, which comes with its own set of issues. AI is fallible, people are unfamiliar with technology, and mistakes happen. What can credit unions and staff do to stay up to date? Let’s find out.
This is the first part of a three-part blog series on issues with AI and how to mitigate them. Today’s blog will focus on AI Hallucinations, what they are and how you can limit their impact.
Hallucinations
An AI hallucination is when an AI provides an answer, product, or output that is not entirely grounded in real data or facts. As with the example above, when an AI hallucinates it creates fake facts, data, or sources.
Hallucinations are a common problem with AI, especially LLMs like Chat GPT or Google’s Gemini. While my colleague knew that the AI response was incorrect, not every hallucination is so harmless, clearly wrong, or so easily detectable.
Does your credit union hire attorneys? There have been numerous cases of attorneys submitting briefs written by AI that contained fake cases or fabricated quotations . Does your credit union use a chatbot? In 2024, a Canadian court ordered Air Canada to pay damages to a customer who was misled by the airline’s chatbot. Beyond chatbots, credit unions can imagine hallucinations in their underwriting process. You might find an AI states that your loan applicant, Michael Baxter Jordan, is a famous basketball player or that he recently won an Oscar (aka “Michael B. Jordan”).
Unfortunately, it does not look like hallucinations are going anywhere for the immediate future. However, here are some things credit unions can do to limit hallucinations and the damage done by them.
- Know where the data is coming from
AI is only as good as the data it uses. While I use Reddit, I would not trust an AI that exclusively uses Reddit. If you have AI underwriting, what is it using to make recommendations and or decisions? Same with chatbots. Credit unions should know where an AI is pulling data from and have limits in place so that it only uses data and sources that you approve.
- Be specific and provide examples
Hallucinations often happen when an AI receives an open-end request. It will find ways to expand upon your request in ways you couldn’t even imagine. When using AI, credit unions should be very specific with their “ask” and keep it as simple as possible. Credit unions can also provide an AI with an example of what you are looking for. Providing a sample helps limit the scope of what the AI will do. If you want AI to update a privacy policy, give it the original privacy policy as an example and be clear you want it to look the same.
- Keep humans in the loop
A day may come when AI replaces humans completely, but it is not this day. Humans are still necessary and can help catch hallucinations. One way to do this is to break up and layer AI activity on top of each other and have human review at crucial junctions. For example, instead of having an AI review and provide a recommendation on a loan workout request, separate the process into separate parts like so:
- Request the AI provide a summary of the member’s history with the credit union
- Human review
- Request the AI provide a summary of the loan and the potential loss of income from modifying the loan
- Human review
- Request the AI provide details on whether members in similar situations that were denied a modification ended up declaring bankruptcy
- Human review
- Request a recommendation
- Human review
Now, exactly how much human involvement and how many separate layers you should have is going to depend on the credit union’s risk tolerance and familiarity with the AI platform in question. You can always start off with more human involvement and more layers and slowly pull back or combine processes as you become more familiar with the AI in question. Also, be more specific than the prompts I wrote above.
- Talk to your vendors
Most credit unions are not developing their own AI. They are relying on vendors to supply AI products. Talk to them and ask them what they do to limit hallucinations. Ask them if there is anything the credit union can do to limit hallucinations. In general, be curious about the product you are purchasing and its effectiveness. Credit unions may want to reference and negotiate certain warranties in your commercial agreements with these vendors that cover issues with AI such as hallucinations.
