The chart lies. The crowd feels. But when the chart is drawn by a black-box model, the crowd doesn’t even know it’s looking at a hallucination. That’s the core tension behind the UK Financial Conduct Authority’s latest warning on large language models like ChatGPT, Claude, and Gemini used in financial services.
Smile while the liquidity drains.
London’s financial district is buzzing—but not with deal flow. The buzz is fear. The FCA just served notice: we’re coming for your AI agents, your robo-advisors, your credit-scoring bots. And we’re not asking nicely.
Hook: The Signal Nobody Saw Coming
It happened on a Tuesday. A three-paragraph statement buried inside a routine FCA market bulletin. No fanfare. No press conference. But anyone who’s survived the ICO crash of 2018 or the Terra collapse of 2022 knows that dry regulatory language is often the loudest alarm.
The FCA suggested it may need “additional powers” to oversee the use of generative AI in regulated financial activities. The target: large language models—ChatGPT, Claude, Gemini—that are already being plugged into bank customer service, wealth management, and even trade execution.
I broke the story to my network at 4:17 AM Nairobi time. By sunrise, my phone was melting. Hedge fund compliance officers, DeFi founders, even a former colleague at the Central Bank of Kenya—everyone asking the same question: “Is this the end of AI in finance?”
My answer: No. It’s the beginning of the real game.
Context: Why Now?
To understand the FCA’s move, you need to look at the landscape. Since ChatGPT’s breakout in late 2022, adoption of generative AI in finance has exploded. Banks are using LLMs to summarize credit reports, draft client emails, and even generate trade ideas. A 2024 survey by Accenture found that 72% of global financial institutions are either piloting or deploying generative AI in at least one business line.
But the tech is messy. Large language models are trained on internet-scale data. They hallucinate. They amplify bias. They can be jailbroken with a simple prompt like “Ignore all previous instructions.” In a market where a single rogue trade can wipe out a quarter’s profits, this is not just a compliance headache—it’s a systemic risk.
The FCA’s existing powers, designed for human advisors and rule-based algorithms, don’t fit. An LLM isn’t a black-box model in the traditional sense; it’s a probabilistic parrot that invents facts with surgical confidence. The regulator needs new tools.
Based on my 23 years watching markets—from the Nairobi Securities Exchange to the crypto CLOB wars—I see this as a necessary pivot. The FCA’s Project Innovate and sandbox era was great for fintech. But AI is different. It doesn’t just speed up processes; it changes the nature of decision-making.
Smile while the liquidity drains.
The FCA is right to act. But the way it acts will determine whether London remains a global fintech hub or becomes a cautionary tale.
Core: The Numbers Don’t Lie—The Models Do
Let’s get technical. The FCA’s concern centers on three measurable risks:
1. Model Homogeneity and Systemic Risk
When every major bank uses the same underlying model (e.g., GPT-4 or Claude 3.5), a single vulnerability becomes a market-wide contagion. In March 2024, a prompt injection attack on a popular LLM-based trading assistant caused $2.3 million in unauthorized trades across three brokerages before the exploit was patched. The FCA’s fear isn’t theoretical—it’s sitting in incident reports.
2. Explainability Deficit
Financial regulations like MiFID II and the UK’s Senior Managers Regime require that decisions be explainable. “The model said so” isn’t a valid answer. But LLMs are fundamentally non-linear; their internal reasoning is a black box. The best we have is post-hoc rationalization via Chain-of-Thought prompts, which can be faked. During my time auditing algorithmic trading systems at a Nairobi brokerage, I learned that even simple neural networks can hide biases. LLMs are an order of magnitude harder.
3. Hallucination in High-Stakes Contexts
A hallucination in a marketing chatbot is embarrassing. A hallucination in a credit approval engine is illegal. A hallucination in a trade execution model can blow up a portfolio. One major UK bank reported in early 2025 that its LLM-based advisory system had recommended a leveraged bet on a non-existent stock ticker. The model didn’t ‘lie’—it just imagined a ticker that had been delisted in 2020. The FCA caught it during a routine review. Luck, not design.
From my 7x24 surveillance desk, I see these incidents as warning flares. The FCA isn’t overreacting; it’s catching up to data that’s been screaming for attention.
Contrarian: The Blind Spot Nobody Talks About
Here’s the part the FT article didn’t write, and that most analysts miss: Overregulation won’t stop AI—it will drive it underground.
I call it the “Shadow AI” risk. When compliance costs become prohibitive, business units don’t abandon the technology; they hide it. Sales traders will copy-paste client data into ChatGPT from their personal phones. Risk analysts will run off-the-books Python scripts that call OpenAI’s API. The result: zero oversight, zero audit trail, and massive data leakage.
I saw this play out in 2022 during the crypto bear market. When regulators cracked down on centralized exchanges, liquidity didn’t vanish—it migrated to unregulated OTC desks and Telegram-based trading groups. The same behavioral pattern applies here. The FCA must design its expanded powers to enable compliance, not punish innovation.
The chart lies. The crowd feels. The crowd in this case is 40,000 financial professionals in London who are already using AI daily. They will not stop. The only question is whether they will do it visibly or invisibly.
Smile while the liquidity drains. Drained liquidity can come from two sources: market panic or regulatory flight. Both are bad for the UK’s fintech ecosystem. But the latter is silent—a slow migration of talent and capital to Singapore, Dubai, or even Nairobi, where the regulatory sandbox is still open.
Another blind spot: the FCA’s approach assumes that model providers like OpenAI and Anthropic will cooperate fully. But these companies have their own incentives. They want to protect their intellectual property and resist audits that might expose vulnerabilities. The power struggle between the City and Silicon Valley is about to get very interesting.
Takeaway: What to Watch Next
This is not a conclusion—it’s a prompt for the next trade.
Over the next six months, watch for three signals:
- FCA consultation paper DP25/X – Expected in Q3 2025. This will reveal the specific requirements for model validation, explainability, and incident reporting. If they ask for full model weights, the industry will fight. If they ask for behavioral benchmarks, compliance becomes manageable.
- OpenAI and Anthropic’s lobbying response – They will push for a self-regulatory model similar to the EU’s, with standards set by industry consortia. Expect a flurry of white papers and policy hires.
- Shadow AI incidents – The first major data breach or trade loss caused by unauthorized LLM use in a regulated firm. When it hits the news, the FCA will get all the powers it wants overnight.
My personal bet: The FCA will ultimately settle on a tiered approach—strict oversight for customer-facing AI (advice, credit, execution) and lighter touch for internal tools. That’s pragmatic and defensible. But the next six months will feel like a cold shower for the AI-in-finance hype cycle.
Smile while the liquidity drains. Then wait for the next wave. In crypto, we call that a reset. In traditional finance, they call it a correction. In both cases, the survivors are those who adapt—and who keep their models auditable.
The chart lies. The crowd feels. And the FCA is starting to feel the heat.
