Tamsin Crossland, Principal AI Architect at Icon Solutions, explores how banks can manage the cost of Artificial Intelligence by optimising data architecture, selecting appropriate models and reducing unnecessary processing.
Artificial Intelligence is entering a more consequential phase in banking. After years of experimentation and proof-of-concepts, banks are beginning to move AI into production and deploy it across the enterprise. That is when a different question becomes unavoidable: what is all of this going to cost?
The economics of AI are unlike traditional enterprise software. Banks are accustomed to relatively predictable costs for licences, infrastructure and support. Large Language Models introduce a fundamentally different model: consumption. Every prompt, response and piece of information fed into a model has a cost attached to it.
At pilot stage, that cost can look negligible. At enterprise scale, it can become substantial. An AI assistant used by a few employees may be inexpensive; the same capability deployed across thousands of staff, customer interactions and operational processes can create a very different financial exposure. One large retail bank I spoke to reported running costs of more than £2 million a month after an AI project went into production.
That makes AI cost management more than an optimisation exercise. It could become one of the defining constraints on how quickly banks can scale AI.
The obvious response is to focus on the price of the model: use a cheaper provider, negotiate better rates or reduce the cost per token. But this risks focusing on the smallest part of a much larger problem. The real opportunity is to reduce the amount of AI a bank needs to use in the first place.
Architecture determines how often a model is called, how much information it has to process and whether an LLM is being used for a task that actually requires intelligence. Those decisions can have a far greater impact on the economics of AI than shaving a fraction off the price of an individual token.
Understanding where AI creates value
One of the most important considerations for banks is recognising that not every part of a business process requires Artificial Intelligence.
Many enterprise workflows contain both judgement-based activities and structured execution. AI is particularly valuable in areas where interpretation, reasoning or analysis are required. However, once a decision has been made, traditional software is often better suited to carrying out the resulting action.
A payment repair process provides a simple example. A failed payment may require analysis to determine why it failed and what action should be taken. This is an area where an AI model can provide significant value because it involves understanding context and making a recommendation.
The subsequent steps – updating records, validating information, resubmitting the payment and notifying operational teams – are deterministic activities. They are already handled effectively by banking platforms and workflow systems.
Using an AI model throughout the entire process creates additional cost without necessarily improving the outcome. It increases the number of model interactions and introduces additional latency into a process that could otherwise be executed efficiently.
The most effective AI architectures therefore separate reasoning from execution. They use AI where judgement is required and rely on established software capabilities where reliability and consistency are more important.
The cost impact of data design
Another significant factor influencing AI expenditure is the quality of information provided to models.
Large Language Models do not only process questions. They process the context supplied alongside those questions. Poor retrieval strategies can therefore create substantial additional costs by providing models with more information than they need.
A banking employee asking about a policy procedure does not need an AI system to process an entire document repository. Yet early implementations often retrieve large volumes of information and pass everything into the model, requiring it to analyse and filter unnecessary content.
This approach increases token consumption and reduces efficiency. The challenge is not simply giving AI access to more data. It is ensuring that AI receives the right data.
This requires strong information architecture. Operational data platforms can provide access to real-time business information. Vector databases can identify relevant sections of documents based on meaning rather than keywords. Graph technologies can help models understand relationships between customers, accounts, devices and transactions.
Each of these approaches reduces the amount of unnecessary information being processed, which directly reduces AI cost.
Why larger models are not always more economical
Another assumption influencing AI investment decisions is that the most advanced model will always deliver the greatest value. In reality, capability and cost need to be balanced carefully.
Some banking use cases genuinely require sophisticated reasoning. Regulatory interpretation, complex analysis and strategic decision support may justify the use of advanced models.
However, many enterprise activities do not require that level of capability. Classification, summarisation, document processing and information extraction can often be completed effectively by smaller and less expensive models.
This makes intelligent model selection increasingly important. The future of enterprise AI will depend less on choosing one preferred model and more on creating systems that automatically select the appropriate level of intelligence for each task.
Using the most expensive model for every interaction is unlikely to be a sustainable operating model.
The importance of avoiding duplication
As AI adoption accelerates, another cost challenge is emerging: duplication.
Different business units often begin developing their own AI solutions, creating separate agents, retrieval systems and evaluation approaches. While experimentation is essential, this approach can create unnecessary engineering effort and increase operational complexity.
Large organisations will need to move towards reusable AI capabilities. Shared platforms, common governance frameworks and standardised architectures will allow successful solutions to be scaled more efficiently.
The objective should not be to create thousands of individual AI applications. It should be to build a controlled enterprise capability where investment can be reused across multiple areas of the organisation.
Creating sustainable AI economics
The long-term cost of AI will not be defined solely by model pricing. It will be determined by the decisions organisations make about architecture, data, process design and governance.
Banks that approach AI simply as a collection of model interactions will inevitably face rising costs as adoption increases. Those that design intelligent operating models will be better positioned to achieve sustainable value.
The most effective AI strategies will combine multiple technologies rather than relying exclusively on large language models. They will use traditional software where it remains the most efficient solution, apply AI where genuine reasoning is required and ensure that information is structured effectively before it reaches a model.
AI will undoubtedly become an important part of banking’s future technology landscape. The organisations that benefit most will not necessarily be those that consume the greatest amount of AI capacity. They will be those that understand how to apply intelligence efficiently.
Managing AI cost is not about using less technology. It is about designing systems that use technology more intelligently. Ultimately, the cheapest AI token isn’t the one purchased at the lowest price. It’s the one your architecture never needs to spend.
