As AI becomes more deeply embedded in financial services, Marc Fernandez, Chief Strategy Officer, Neurologyca, says the growing divide between machine-driven decisions and unpredictable human behaviour could create a new challenge for banks, regulators and customers.
When US Treasury Secretary Scott Bessent and Federal Reserve Chair Jerome Powell reportedly gathered some of the country’s largest bank CEOs behind closed doors to discuss the risks posed by advanced AI systems, it was something of a turning point.
The finance sector has been among the most prolific when it comes to AI adoption, but models are advancing so quickly that the risks they may pose – or eventually pose – largely took a back seat.
Much of their discussion centred on cybersecurity, particularly the emergence of models capable of identifying and exploiting software vulnerabilities at a scale and speed that would have seemed unthinkable only a few years ago. I think that concern is entirely justified. Financial institutions are already dealing with AI-generated phishing campaigns, increasingly sophisticated fraud and attack surfaces that are becoming harder to monitor as automation spreads across the industry.
But the more consequential risk may have less to do with AI attacking the financial system and more to do with the financial system slowly drifting out of sync with the people it’s meant to serve.
Banking infrastructure has always been built around assumptions about how people make decisions. Credit scoring, fraud detection, underwriting, trading systems, even customer onboarding flows all rely on behavioural patterns remaining relatively stable and interpretable.
What’s interesting about AI is that it’s completely changing the pace and nature of those interactions. Systems are now capable of acting autonomously, processing enormous volumes of information and moving decisions forward continuously, while the humans interacting with them remain emotional, inconsistent, hesitant and prone to changing their minds halfway through the process.
While a human agent can respond to these emotional cues, an AI agent cannot. That’s creating a ‘human blind spot’ that the industry isn’t talking about enough.
Banking was designed around predictable human behavior
Financial systems have never been purely mathematical, even if the industry likes to present them that way. Underneath every lending model, fraud engine and risk framework sits a set of assumptions about how people behave under pressure.
A customer applying for credit might hesitate before accepting terms; an investor might pull back from a decision despite the data pointing in one direction or a borrower who suddenly starts acting differently may signal stress long before a payment is missed. Experienced professionals learn to spot these shifts instinctively because human behaviour has always been part of the operating environment of finance, whether institutions formally acknowledge it or not.
The problem is that most AI systems don’t interpret those signals. They process inputs, generate outputs and continue moving forward according to the logic they were given at the start of the interaction. Meanwhile, the human on the other side may already be reconsidering the decision entirely.
Confidence can disappear halfway through a transaction. Confusion can creep into an onboarding process. A customer may stop trusting a recommendation despite initially engaging with it positively. None of that nuance is visible to the system itself, no matter how intelligent or capable the model.
As AI becomes more deeply embedded across banking workflows, from customer service to underwriting and investment support, the industry is entering unfamiliar territory where machines operate at enormous speed while the human context surrounding those decisions remains fluid and unpredictable – and that gap is only going to widen.
The dangerous gap between AI outputs and human intent
This is why the risk of AI in finance isn’t just cybersecurity-related. The financial sector is rapidly deploying systems designed to optimise for speed, efficiency and predictive accuracy, but very few of those systems have any real understanding of whether the human being involved is still aligned with the direction the process is taking.
AI can recommend a financial product, flag suspicious activity, calculate exposure or surface an investment strategy, but it usually has no awareness of whether the person receiving that recommendation has become uncertain, overwhelmed, distracted or hesitant halfway through the interaction.
I think that disconnect matters more than many of us realise. Financial decisions unfold across sequences of interactions where confidence, risk tolerance, stress and intent can evolve over time. People second-guess themselves constantly, especially when money, risk or uncertainty are involved.
An investor may initially appear confident before becoming cautious as market conditions shift or a customer applying for credit may rush through forms despite not fully understanding the implications. Likewise, a fraud system may escalate perfectly legitimate behaviour simply because it cannot distinguish between urgency, confusion, stress or malicious intent.
Yet most AI systems continue processing as though none of those changes are happening because they are still operating off the original inputs and assumptions. They’re logic engines, not emotional ones.
At scale, this starts to influence the quality of financial outcomes themselves. The industry often talks about optimising for the ‘best’ recommendation or prediction, but in practice the real-world outcome depends on whether the human being involved actually trusts, understands and follows through on the decision.
That ‘de-sync’ between banking and human intent may become one of the defining challenges of financial AI over the next decade.
Measuring intuition in an AI-driven financial system
One of the more interesting things happening inside financial institutions right now is a growing recognition that intuition still plays an enormous role in decision-making, even in highly data-driven environments.
According to one study involving traders across four major investment banks, high-performing traders relied more on intuition and emotional awareness than data-driven insights, especially when operating under pressure. The same applies to customers – ‘gut feel’ is just as important as any other data point. AI is optimised for logic and prediction, not for interpreting the shifting human context surrounding a decision.
Those instincts have always been viewed as intangible human qualities that sit outside the reach of technology. And AI has exposed the limits of that assumption because systems are now influencing increasingly high-stakes decisions without any visibility into the confidence, hesitation or cognitive state of the people interacting with them.
That’s starting to open the door to a different way of thinking about financial AI. Instead of focusing purely on generating better outputs, attention is beginning to shift toward understanding how humans respond to those outputs in real time.
Signals like hesitation, pace, disengagement, inconsistency or sudden changes in behaviour can potentially be treated as meaningful contextual data rather than noise. In practice, that could allow systems to distinguish between informed intuition and emotional reaction, identify overconfidence before a critical decision is made or adapt how information is presented when a user appears uncertain.
We need to build systems that remain sensitive to the reality that financial decisions are made by people whose thinking evolves moment by moment, not by static inputs on a screen. The next generation of financial systems may need to continuously recalibrate interactions as human confidence, intent and cognitive state evolve.
Financial resilience depends on keeping AI aligned with people
Fraud, cyberattacks and adversarial AI will continue dominating headlines and understandably so. Financial institutions have spent decades building defences against external threats and they’ll continue adapting as those threats evolve.
But the harder challenge may be internal. As AI systems become more autonomous and more deeply embedded across financial workflows, the industry is beginning to rely on technologies that can process information at extraordinary speed without necessarily understanding the human context surrounding the decisions they influence.
That creates the possibility of systems becoming operationally efficient while gradually drifting away from the people they are meant to support. The quality of financial outcomes increasingly depends not just on what systems recommend, but on how humans respond to those recommendations over repeated interactions.
The conversation regulators are now starting to have is an important one because it moves the focus beyond whether AI can generate accurate outputs and toward whether those systems remain aligned with real human outcomes over time.
Financial stability has always depended on more than raw computational power. It depends on trust, confidence, judgement, hesitation, caution and the messy realities of human behaviour that don’t fit neatly into structured datasets.
AI will almost certainly become foundational to the future of banking, but the institutions that navigate this transition successfully may be the ones that recognise something the industry has historically underestimated – that people cannot be read like machines.
