AI adoption in banking: 5 “hidden” risks
Misgivings over the use of AI in banking have given way to its ubiquity across all areas. But while the power of AI is undisputed, human oversight and intervention is not just necessary – it is a regulatory requirement in most cases. But its effectiveness is only as good as the domain knowledge these humans can bring to bear on their activities.
Here are 5 “hidden” areas where the role of humans remains critical to mitigate risk in AI-driven decision-making.
This article is also available in podcast/video form. Watch the video below from our YouTube channel, or follow The Intuition Finance Digest on Spotify, Apple Podcasts, or Amazon Music.
1. Governance and Accountability
The power of AI in terms of information processing means there is an understandable tendency for people to rely on the technology as if its output was unimpeachable. Given that assumption, the natural approach of banking practitioners is to absolve themselves of responsibility for its decisions.
But regulators are not accepting the argument that “the model did it” and management remains on the hook for decisions made with AI tools.
This is a key principle. Governance and accountability risk in AI is acute for the reason that AI decisions often sit between traditional control categories. For example, a chatbot may be partly customer communication, partly advice support, partly complaints handling and partly data processing.
With AI decision-making, personal ownership must be explicit, otherwise risks fall between departments.
Ultimately, someone must answer: “Would I be comfortable explaining this decision to a regulator?”
It remains for human managers to:
approve use cases
establish limits
determine accountability
challenge unexpected outcomes
Depending on jurisdiction, that requires detailed appreciation of areas such as consumer duty, MiFID, AML, data privacy, and conduct regulation, together with risk management concepts such as three lines of defense, risk appetite, and escalation procedures.
2. Model Risk
Model risk is nothing new: it already exists in credit scoring, fraud detection, stress testing, pricing, capital modeling and trading analytics. But generative AI adds new risks: hallucination, inconsistent outputs, prompt sensitivity, weak source attribution, retrieval errors and plausible but wrong explanations.
One of the notable characteristics of banking and finance is that rare events tend to be the most material. Financial crises, liquidity spirals, cyber incidents and market crashes are low-frequency, high-impact events.
Ironically, models trained mainly on normal data may be least reliable when judgment is most needed. Humans provide the reasonableness testing and challenge to AI models.
So, for the credit officer that means understanding concepts such as leverage, cash flow, collateral, covenant structures, and industry cycles
For the investment officer valuation, market structure and an appreciation of economic cycles remains essential.
3. Data Risk
AI is only as good as the data it is fed, but banking data is fragmented, sensitive and heavily regulated. The result: poor AI outputs.
Data-related issues are among the top perceived AI risks, and the Financial Stability Board (FSB) has warned that common data sources and models can create concentration and herding risks.
Bad data frequently looks plausible.
Humans assess whether data makes commercial sense.
On the operations side this means having detailed knowledge of source systems and data lineage
Critically, that must be complemented with product knowledge and an appreciation of expected customer behavior.
So, for example, when AI identifies a large increase in deposit balances, an experienced treasury manager is on hand to assess whether this is down to a reporting change; if the nature of the balances are temporary and whether there is a genuine improvement in the liquidity position.
4. Conduct and Consumer Protection Risk
AI can improve service, but it can also compound poor treatment of customers at scale.
In retail banking, the main conduct risks are mis-selling, unsuitable recommendations, exclusion, unfair pricing, poor vulnerability handling and misleading explanations.
This is perhaps the area where human judgement can make most difference, practiced as it is in recognizing the nuances behind human conditions such as vulnerability, distress, coercion, confusion, and hardship.
5. Bias and Fairness Risk
Discrimination can be an unintended outcome of AI.
Models may use variables that correlate with age, gender, ethnicity, disability, income instability, geography or digital access. In banking, this can affect lending, pricing, fraud controls, collections, authentication and service access.
Bias testing must go further than technical fairness metrics. It should include customer outcome testing, vulnerable-customer impact analysis, and review by people who understand banking products and customer behaviour.
That requires fair lending and conduct specialists with knowledge of discrimination law, lending practices, customer demographics, customer outcomes, and regulatory expectations.
![[Feature Image] Compliance From obligation to business value](https://www.intuition.com/wp-content/uploads/2026/06/Feature-Image-Compliance-From-obligation-to-business-value-700x441.webp)
Frequently asked questions
Why does human oversight still matter in AI-driven banking decisions?
Human oversight still matters because regulators do not accept the argument that the model made the decision. Management remains responsible for decisions made with AI tools, especially where AI activity crosses control categories such as customer communication, advice support, complaints handling, and data processing. Human managers must approve use cases, set limits, define accountability, and challenge unexpected outcomes before risks fall between departments.
What is the governance risk of AI adoption in banking?
The governance risk is that AI decisions can sit between traditional control categories, making ownership unclear. A chatbot, for example, may involve customer communication, advice support, complaints handling, and data processing at the same time. Without explicit accountability, risks can fall between departments. Human managers need enough regulatory and risk knowledge to explain AI-driven decisions to regulators.
How does generative AI change model risk in banking?
Model risk already exists in areas such as credit scoring, fraud detection, stress testing, pricing, capital modeling, and trading analytics. Generative AI adds risks such as hallucination, inconsistent outputs, prompt sensitivity, weak source attribution, retrieval errors, and plausible but wrong explanations. This makes human challenge important, especially when rare but material events require judgment beyond normal training data.
Why is data risk a major issue for AI in banking?
Data risk is a major issue because AI outputs depend on the quality of the data it receives, and banking data is often fragmented, sensitive, and heavily regulated. Bad data can still look plausible. Human experts are needed to assess whether data makes commercial sense by understanding source systems, data lineage, product behavior, and expected customer activity.
How can AI create conduct and consumer protection risk?
AI can improve service, but it can also scale poor customer treatment. In retail banking, risks include mis-selling, unsuitable recommendations, exclusion, unfair pricing, poor vulnerability handling, and misleading explanations. Human judgment is especially important because experienced practitioners can recognize customer vulnerability, distress, coercion, confusion, and hardship in ways that automated systems may miss or misread.
How should banks assess bias and fairness risk in AI?
Banks should assess bias and fairness risk by looking beyond technical fairness metrics. AI models may use variables that correlate with age, gender, ethnicity, disability, income instability, geography, or digital access. Testing should include customer outcome reviews, vulnerable-customer impact analysis, and input from specialists who understand fair lending, conduct regulation, banking products, and customer behavior.
Build the banking fluency your teams need to use AI with confidence, sound judgment, and regulatory awareness across modern financial markets with Intuition Know-How.
Click here to learn more, or fill out the form below.
