January 8, 2025

AI in Model Risk Management: A Guide for Financial Services

Share

Our modern financial system of banks, stock markets, investment firms, derivatives, and smartphone apps would be all but unfathomable to the 15th-century Italians who invented double-entry bookkeeping.

That complexity is exactly why today’s financial institutions are leaning on advanced statistical models and artificial intelligence to assist with credit scoring, fraud detection, portfolio management, and risk management.

In 2008, quantitative models played a major role in the financial crisis—a fact that regulators such as the Fed and the Bank for International Settlements (BIS) took very seriously. These models posed a systemic risk, but very few people knew how they worked or how to limit their power.

The frameworks that regulators introduced to deal with quantitative model risk in the 2010s are now being applied to AI model risk management. Even though AI models share a lot of statistical DNA with earlier quantitative models, they differ in fundamental ways.

AI models can learn and adapt to new data in ways that previous financial models couldn’t. This dynamic element introduces new risks, such as explainability, bias, data governance, and more. Financial institutions and the vendors who serve them need to treat AI model risk management (MRM) as a core strategy, not a box to check.

In this article, we’ll take a deeper look at why AI model risk management is important, the challenges it poses, and the current regulatory environment. We’ll also wrap up with a look at what the future holds and how institutions can get ahead of the curve.

How Does AI Fit into Model Risk Management?

When bank regulators began requiring institutions and investment firms to manage the risk associated with their quantitative models, it’s safe to say they couldn’t see AI barreling down the metaphorical highway in their rearview mirror.

Previous models were static enough to undergo rigorous testing, validation, and monitoring without creating an unholy amount of work for the teams deploying those models. If the model passed the tests, you could deploy it and just peek in from time to time in case something went sideways.

Now, each AI model is a different animal—an animal that can digest mountains of data, learn on its own, and provide novel insights based on those learnings. Many of the current models have been fed highly controlled data sets and relied on humans to help shape the training process.

But even this traditional AI modeling isn’t as far as AI can go.

Generative AI (GenAI) takes MRM to a new level. Based on neural networks and often equipped with natural language processing (NLP) capabilities, GenAI is able to learn on its own and create outputs that have far more variability than previous models.

Fitting AI into the current best practices for model risk management isn’t straightforward. Predictive models that focus on narrow datasets can be tested, validated, and monitored in a repeatable fashion.

GenAI models are far-reaching in their appetite for data and far less predictable in what they produce. As such, they require far more documentation, testing, and monitoring to ensure they’re free from biases, egregious hallucinations, and faulty construction.

Financial institutions with existing MRM organizations and processes will need to rethink their approach if they want to take full advantage of AI and GenAI. However, these dynamic neural network models will pose extreme risks if they aren’t managed properly.

The Importance of AI Model Risk Management

Prior generations of models were designed for specific challenges. If you were using a quantitative model to suggest how high or low the price of an asset might go, the risk was largely limited to that situation or trade.

The methodology of the model was also reasonably transparent, at least to the quants and statisticians who designed it.

In the case of AI, the methodology of its decision process is much harder to discern. Hence the tendency to refer to them as “black box” models. AI models also represent a significant breakthrough in technology—their power and multi-functionality give them an edge in certain types of high-stakes decision-making.

By deploying these types of models at scale, financial institutions may be wading into much deeper water than they expected. If the outputs and downstream decisions facilitated by AI models aren’t validated and monitored closely, they could lead to severe consequences such as:

Inaccurate predictions or analysis.
Lost opportunities and wasted resources.
Compliance breaches and penalties.
Harm to brand reputation.

AI models may differ from conventional models, but regulators are every bit as keen to see financial institutions performing due diligence on them. Documentation, testing, validation, monitoring, and governance will require more sophistication to deal with the complexity and scale of the situation.

On the upside, rigorous MRM can help your institution build trust with stakeholders, avoid compliance problems, and contribute to responsible AI model development.

Challenges of AI Model Risk Management

Some of the challenges for AI MRM are similar to conventional MRM, but at a much higher degree of difficulty. Here are the big issues you should be tracking and mitigating in your organization’s MRM .

Data Quality

AI models have a much greater appetite for data. Without sufficient, cleansed, standardized data, no model will perform.

If fed properly, that data greatly enhances their accuracy. On the other hand, giving an AI model terabytes of bad data will only refine its ability to reach erroneous and biased conclusions.

These errors can have terrible consequences if used for tasks such as underwriting decisions or insurance claims. The best remedy is data quality control and good governance for your systems and processes.

Bias in AI Models

If the training dataset for a model is biased, the model will express that bias.

And like humans, the AI model will have no idea that it’s returning a biased result. This includes biases we associate with individuals (such as a preference for a certain gender) and societal biases (such as preference for a certain nationality).

Model developers can lessen the tendency for bias by thoroughly evaluating training data for potential biases. They also need to examine the algorithms that govern decision-making within the model to look for additional biases.

Deploying unbiased AI models is the most ethical and compliant approach, especially when dealing with credit and risk assessments.

Explainability and Transparency

The obscure and complex ways that AI models operate have driven the need for explainable AI (XAI) so that regulators and stakeholders can grasp what’s happening in these “black box” models.

Institutions must be able to justify the predictions and decisions of their AI models in order to satisfy internal and external audits. If an AI model’s output is challenged in court, the need for a coherent explanation is even greater.

Ethical Considerations

While the most basic ethical considerations are enforced by law, there are many ethical issues that you should consider on top of what the law says.

How does your model deal with the rights of individuals? Is it fair and unbiased?

Does it comply with relevant laws such as the Equal Credit Opportunity Act and anti-discrimination regulations?

These are questions you want to answer before the regulators ask you.

Regulatory Compliance

Although AI models currently fall under the jurisdiction of the Fed and SR 11-7 for financial institutions in the U.S., these rules weren’t created with AI in mind.

Europe recently introduced the EU AI Act, which will create ripple effects far beyond the borders of the eurozone. Much of the existing regulation coming from the Prudential Regulation Authority (PRA), the Bank for International Settlements, and the European Central Bank (ECB) is being applied to AI models, regardless of how well they fit.

Your MRM organization and processes must comply with current regulations and be nimble enough to adopt new regulations as they get published.

Special GenAI Considerations

The difference between conventional quantitative models and AI is real, but not always extreme. GenAI, on the other hand, marks a leap forward. GenAI models use larger datasets and rely on the formation of neural networks to analyze and generate responses.

These are definitively “black box” models that resist simplification and explanation in everyday terms. It’s also complicated to uncover biases that were introduced during training.

Institutions cannot afford to treat MRM for GenAI as an extension of existing MRM practices. It requires a much more rigorous process of development, documentation, testing, and monitoring to get it right.

MRM and Explainable AI

If you could peer into the human brain with a microscope, it would be highly transparent, but it wouldn’t be easily explainable. You wouldn’t have a deeper understanding of how neurons help make decisions or why a certain process might be biased.

That’s why XAI is such a vital aspect of AI model risk management: regulators and stakeholders need to both see how the models behave and why the models return certain outputs and not others.

Credit scoring and loan underwriting are just two areas where regulators will expect institutions to provide a coherent defense for why the model decided the way it did. Stakeholders will also want explanations for why AI should be trusted to make high-stakes business decisions.

Explainability techniques should include, but aren’t limited to:

You don’t have to be an expert to provide explanations. Working with a vendor such as ValidMind can make the whole process much easier.

Core Components of AI Model Risk Management Framework

If you’re working from an existing MRM framework, that’s great news. The foundational principles will serve you well when it comes to AI MRM. However, there are clear differences and expansions that you should pay attention to.

1. Model Development

The sheer scale of AI models requires an intense level of data hygiene and governance. Their performance is contingent upon getting quality data that supports the business objectives for the model, as well as ensuring fairness and regulatory compliance.

XAI starts during the development process and includes proper documentation to justify model construction and outputs later on.

If you don’t have the internal expertise to properly explain and document your AI models, then consider bringing in a vendor who can streamline the process.

2. Model Testing

Testing your AI models is integral to the development stage, but proper testing should happen throughout the model’s lifecycle. Performance testing tells you if the model is behaving the way it was designed, while stress testing reveals how the model handles adverse scenarios that are beyond its stated scope.

The best way to approach testing is to use a dedicated MRM platform.

If your model lifecycle is managed in one place, it’s much easier to run batteries of tests whenever you need to. Testing is also part of the validation phase, but needs to be handled independently from initial development.

3. Model Validation

Validation for AI models will only work if it accounts for the dynamic and variable nature of the model itself.

Statistical models emphasize consistent outputs within an acceptable range. But AI models learn and adapt to new data, meaning that over time they can produce very different outputs from the same prompt or stimulus.

Your AI model validation process needs to assess the model for algorithmic bias and fairness. Proper backtesting is also a vital part of validating an AI model because it can reveal issues prior to deployment.

The best case scenario is that your validation process strikes a balance of meeting business objectives for the model, including novel insights, while complying with regulations that seek to limit hallucination or wildly divergent outputs that could lead to costly ripple effects.

4. Model Monitoring

AI and machine learning (ML) models can drift over time—they integrate new data, develop new connections, and arrive at new conclusions. Continuous monitoring of the model should include its performance, accuracy, and data integrity.

Periodically the model should be revalidated to ensure all the original constraints and objectives are met. Establishing the right cadence for revalidation will help your institution flag issues in the model before they lead to regulatory penalties or reputational harm.

5. Model Governance

Regulators won’t dictate every policy and procedure for your AI MRM workflows, but they will hold you accountable for the processes you put in place. Healthy governance should include clear roles, responsibilities, reporting structures, and committees for managing model risk.

Your governing structure helps to maintain proper documentation and transparency for how your institution manages model risk and ensures ethical practices at every stage. Proper checks and balances will make all the difference in your next audit.

New AI Regulations That Impact MRM

The need for MRM became painfully clear in the wake of the 2008 financial crisis. AI models haven’t been the focus of a comparable economic crisis, but that hasn’t stopped governments and regulators from stepping in.

The rapid advancement of AI models and their unprecedented ability to transform entire industries has put many people on their guard.

How will we prevent AI from causing irreversible harm? Who is responsible when an AI model does something wrong? Should we trust AI models more than we trust human decision-making?

Here are some of the current rules for how institutions are expected to manage model risk, as well as the consumer data that is used to train them.

The EU AI Act

Presented as the world’s first AI law, the EU AI Act was first released in the summer of 2023. It breaks AI systems into four tiers of risk:

Unacceptable risk: These systems are banned outright, including models that feature cognitive manipulation, certain forms of biometric identification, and social scoring.
High risk: Systems that could negatively affect fundamental rights and safety, including credit scoring and financial services. Institutions must document how models function and submit them to bias detection and fairness testing. High-risk models need continuous monitoring and periodic revalidation.
Limited risk: GenAI systems such as ChatGPT fall into this category and may be subject to additional scrutiny.
Minimal risk: These systems pose no appreciable risk and are exempt from most AI-specific regulations.

The EU AI Act was written to address risks specific to AI, define unacceptable AI risks, list high-risk applications, and set requirements for anyone deploying or providing high-risk AI models.

It seeks to establish clear obligations for AI companies and users, as well as reduce the compliance burdens for smaller businesses. However, failure to comply with the AI Act will result in significant penalties.

GDPR and Data Governance in AI

The General Data Protection Regulation (GDPR) is a privacy and security law introduced by the European Union and it affects anyone who targets or collects data from people in the EU.

This gives the law some jurisdiction over AI models when those models collect personal data:

AI/ML models that fall under the GDPR must track and audit the data used for training models, especially if it’s sensitive or personal in nature.
The law gives EU citizens a right to explainability, meaning that institutions and developers must provide explanations for how the AI model makes decisions. Compliance is much simpler if XAI tools are embedded into the validation and monitoring phases of the model’s lifecycle.

As we’ve mentioned, AI/ML models are extremely data-hungry, and the consumer data available to financial institutions is, by definition, personal and sensitive. You need a robust data governance regime to satisfy both GDPR and MRM best practices.

U.S. Regulatory Focus: SR 11-7 and AI Model Expansion

SR 11-7 is the gold standard for MRM, especially in the U.S. But it wasn’t designed with AI in mind.

Currently, the U.S. financial regulators, including the Fed and Office for the Comptroller of the Currency (OCC), are considering additional rules for AI models to ensure fair lending and consumer safety.

The principles of SR 11-7 remain effective and applicable for AI models, provided that institutions account for AI-specific issues such as model drift, algorithmic bias, and explainability.

Regulators are paying extra attention to AI models used in lending. Compliance with fair lending laws is an absolute must if institutions are going to make the most of AI models and avoid penalties for lending violations.

Institutions with mature risk functions may still struggle with the AI-specific requirements of the EU AI Act, GDPR, and potential expansions to SR 11-7. The spirit of these laws is to create an economic system that embraces the power of AI and ensures transparency, fairness, and accountability for everyone.

The Future of AI Model Risk Management

We’re in a moment of technological upheaval. Only time will tell if the rapid development of AI compares to other pivotal moments in history, such as the age of the locomotive or the harnessing of the atom.

The cutting edge of AI development is moving forward at a blistering pace. Here’s five trends we see on the horizon for AI models and the institutions that deploy them.

1. GenAI and NLP Will Continue To Evolve

Financial institutions will integrate GenAI and NLP models more deeply into their technology stacks. They will have the capability to analyze larger datasets, improve prediction accuracy, and generate comprehensive (and comprehensible) reports with very little oversight.

NLP models will allow institutions to comb through content-heavy sources to inform financial decision-making.

Armed with these new capabilities, institutions will be able to anticipate new risks and capitalize on new opportunities. They will also require continuous improvements to MRM practices.

2. Reinforcement Learning for Dynamic Risk Management

Reinforcement learning (RL) is an exciting development that promises to help AI models improve their decision-making by learning from their environment and weighing short vs. long-term objectives. RL models can take feedback from the market or other sources and continually optimize for better outcomes.

While RL models promise to transform investment portfolios and fraud detection, they are extremely complex and difficult to explain.

Institutions need to be proactive with their XAI processes if they’re going to use RL and stay within the regulators’ good graces.

3. Ethical AI Will Become Central To AI Development

When a technology is in the early phases of development, the people who design and build it are testing to find out what it’s capable of. As we move out of this discovery phase with AI, the importance of ethical AI will grow.

Institutions and model developers will need to operate within explicit ethical frameworks as they innovate and comply with MRM regulations. It’s far better to investigate and establish your organization’s ethics around AI now, before the regulators (and shareholders) make it mandatory.

4. Increased AI Adoption in Emerging Markets

Digital technology has allowed many developing countries to leapfrog the industrial development that developed countries experienced. That means they’re able to deploy AI in markets that are smaller and less mature.

This will likely be a source of incredible financial opportunity, but it will also present novel dangers as well.

Institutions that operate in emerging markets should do everything they can to comply with local regulations and implement AI models in safe, equitable ways.

5. AI-Powered Model Risk Management Platforms

We’ve said it before, and we’ll keep saying it: manual processes are incapable of addressing the scale and complexity of conventional MRM and AI MRM. You need tools that are as advanced and capable as the models you’re deploying.

Platforms such as ValidMind give you a command center for your MRM organization, including testing, validation, monitoring, and governance.

You don’t need to reinvent every aspect of MRM to fit your organization; you need tools that can be tailored to your business and help guide your compliance efforts. By partnering with an expert in MRM and AI MRM, you can ease the burden on your team and increase their effectiveness.

ValidMind: Your Partner in AI Model Risk Management

If there’s a central theme to this article, it’s that AI model risk management is a merging of two realities: the power of technology and the importance of humanity. AI is spurring rapid innovation across many industries, but society isn’t willing to jump in with both feet.

Consumers and governments are right to ask for guardrails on the AI-superhighway. Institutions and businesses are also right to use AI to create new value and efficiencies.

It’s possible to balance automation and human oversight. ValidMind lets MRM stakeholders and financial institutions focus on what they do best, without compromising their compliance or ethical obligations.

MRM should be a source of confidence and added business value, not pain and fines. Request a demo to find out how ValidMind can help.