In the fast-paced world of financial services, banks and other financial institutions are increasingly leveraging artificial intelligence (AI) and machine learning (ML) solutions to enhance their business operations. As these technologies become more sophisticated, the use of third-party model vendors has become a popular avenue for institutions seeking to access the latest advancements in AI/ML models without the need to develop these capabilities in-house. This adoption is particularly evident in areas such as anti-money laundering (AML), asset-liability management (ALM) models, operational efficiency and process automation models, and AI-powered customer service operations models such as chatbots.

However, managing third-party models introduces a unique set of challenges for banks, particularly from the perspective of model risk management (MRM). Regulators globally are clear on the responsibility for institutions to manage vendor model risk: 

• In the USA, according to SR 11-7 guidelines, financial institutions are expected to validate their own use of vendor products [OCC 2011-12, p.15], which means that the same rigorous MRM principles apply regardless of whether a model is developed internally or by an external vendor.
• In the UK, the PRA’s SS1/23 states that firms should […] satisfy themselves that the vendor models have been validated to the same standards as their own internal MRM expectations [PRA SS1/23, p. 17].

While the regulatory mandate is clear, it can be very difficult for MRM teams to access model development evidence from vendors due to intellectual property concerns, which limits the ability to validate the model. Vendors often employ complex and sophisticated models which the bank doesn’t necessarily have the skills to validate. Moreover, vendor models often embed dependencies on external data sources (e.g., for model training or as part of a core capability) which the bank cannot access, further increasing validation complexity. Finally, delineating communication processes and roles/responsibilities between the bank and vendor can be challenging, e.g. establishing how key model findings will be communicated and who will be responsible for tracking remediation. 

So, what can banks do to effectively manage the risks associated with third-party vendor models? Here are some essential recommended practices:

• Track your vendor models: Banks must maintain an up-to-date inventory of all vendor models in use, categorizing each according to its risk level. This helps in prioritizing oversight and validation efforts.
• Collect documentation and evidence: To ensure proper validation, banks need to have mechanisms to collect comprehensive documentation from vendors, including details on model development, independent validation reports, and performance monitoring data. Such documentation provides insights into the model’s design, its testing and performance outcomes, and the appropriateness of the data used by the vendor. MRM teams may collect this evidence by collaborating with vendor or supplier management teams as part of the vendor onboarding process. Platforms such as ValidMind can also facilitate the evidence collection process by offering an automated approach which protects proprietary vendor information.
• Monitor model performance: Monitoring the vendor model for changes and making sure that the bank is made aware of updates to the model in a timely manner is critical. This may involve embedding service level agreements (SLAs) in vendor contracts to ensure the bank stays informed of any modifications made by the vendor, and ensuring a robust process is in place to assess the impact on the model’s performance and risk profile.
• Establish clear roles, responsibilities, and communication processes with model vendors: MRM teams can work with business teams to clearly delineate roles & responsibilities between the bank and the vendor when it comes to timely information sharing, findings remediation, and to put in place the appropriate communication channel with the vendor in relation to model risk information. Early collaboration between 1st and 2nd lines of defense is essential here, to ensure there are no slowdowns in the validation process and missing information.

These are some of the practices that financial institutions can adopt to proactively integrate third-party models in their MRM framework. Broader considerations should also encompass rigorous documentation and validation templates to support evidence collection from vendors, and governance mechanisms to monitor and respond to model changes. By embracing these practices, banks can harness the power of third-party AI/ML models, while ensuring that the associated risks are effectively managed.

To learn more about how ValidMind can help financial institutions extend their MRM capabilities to vendor models, get in touch! 

Sources

• OCC/Fed Supervisory Statement on Model Risk Management (SR 11-7) 
• FDIC FIL-22-2017, Supervisory Statement on Model Risk Management
• Bank of England, PRA SS1/23 Model Risk Management Principles for Banks