Rebooting AI in the Age of Hallucinating Agents

A former client – now a good friend – and Lars exchanged holiday greetings this morning. He observed that conversations between them have become far less frequent than they were during the years when he managed billions in fixed-income for a large institution, and Lars ran a fixed-income trading desk on Wall Street as one of the original “bond vigilantes.”

When your AI agent recommends a $50M acquisition, underwrites a million-dollar credit line, or drafts language a regulator will scrutinize, a question becomes unavoidable: Who is liable when the model is wrong?

Not the model. Not the cloud provider. The person who deployed it.

During this year's Thanksgiving festivities and gatherings, the BRC FinTech team had several conversations about the current state of Large Language Models ("LLMs") and the future of Artificial Intelligence ("AI"). Almost everyone expressed wonder at how "good" and relatively complete model responses have become, often giving near PhD-level answers that sometimes sound more knowledgeable and confident than their information retrieval and knowledge warrants. The conversations then devolved into various LLM prompting strategies and advanced learning discussions.

As we start the year of the 2024 American federal elections, there is significant uncertainty. One thing though is virtually certain: the year will be much more volatile than what is currently being priced into the Capital Markets.

There are considerable geopolitical issues, at least one of which is likely to shift economic outlooks. Wars are currently raging in Ukraine and Gaza with other tensions in the areas around the Red and South China Seas. These conflicts will require economic and political resources, both of which seem to be in more limited supply today than earlier.

As the day starts on this Thursday, the 10-year TSY and 30-year TSY yields are at 4.97% and 5.08% respectively. If these yields also are where these benchmark securities close this afternoon, they would represent yet another closing high yield level since 2007.

October 4th 2023 has already been a crazy day in the global capital markets. As Brass Rat Capital ("BRC") wrote in the 2023-09-04 WILT post, this FinTech skunkworks sensed the American economy and capital markets were about to go through some volatile and relatively negative times.

A very high percentage of the contacts in the Brass Rat Capital ("BRC") electronic Rolodex appear to use Google's Gmail service. BRC (or its predecessors) has done so for more than a dozen years. If you use the Google ecosystem, including contacts, Gmail, Docs, YouTube, and Maps, PLEASE, PLEASE refrain from using the new Google AI chatbot that allows it to act as a personal assistant. The same goes for the OpenAI, Microsoft, and Meta chatbots that supposedly can access both your personal information and web searches. DO NOT USE these AI Assistants yet.

If one is a capital markets participant, surely one recalls the 1979 Saturday Night Massacre. Correct? Most will not since many were not yet even born. Brass Rat Capital suggests that one pause now to conduct a deeper dive to understand the "Nifty Fifty" period and what happened during the 1973 and 1979 oil shocks that resulted in the October 1979 Saturday Night Massacre and ultimately doomed the Jimmy Carter Presidency. Does one appreciate the horrific gloom during the period that the actor Ronald Reagan started his Presidency?

This last week of September increasingly appears to be a turning point in the American capital markets. As Brass Rat Capital ("BRC") wrote about recently, longer-term interest rates are now rising. The equity market is breaking down in a technical sense. Prices in the energy sector are strongly increasing, and the U.S. Dollar is amazingly strong.

Perhaps the most major problem with Large Language Models ("LLMs") and hence generative AI is when the model(s) generate factually incorrect text. The Gen AI now refers to this phenomenon as producing "hallucinations." As a result, a key AI research area has become "explainability." In short, how does a human being understand how the model came up with "that" result?