How Small Banks May Avoid BSA AML Validation Setbacks

The Federal Reserve[i], the Office of the Comptroller of the Currency[ii], and the Federal Deposit Insurance Corporation[iii] have all published guidance explaining how risk managers should meet Bank Secrecy Act and Anti-Money Laundering compliance requirements.

As more banks started using BSA and AML models for operational efficiency, the need for effective model risk management framework emerged.

SR 11-7[iv], or the Guidance on Model Risk Management by the Fed and OCC, outlines the regulatory expectations on banks’ model risk management framework, including what elements are subject to required model validations. Model risk management expectations for BSA and AML models were further clarified in SR 21-8[v], or the Interagency Statement on Model Risk Management for Bank Systems Supporting BSA and AML Compliance.

Although these guidelines are for banks of all sizes, every bank faces a unique situation. Banks with different asset sizes work with different borrower profiles and resources. For instance, a large conglomerate may extend their credit lines with global banks whereas a local business would work with a regional bank serving the community in that region. This requires banks to assess their idiosyncratic compliance risk.

Smaller banks – banks with under $10 billion in assets – tend to leverage BSA and AML vendor models as these solutions help them detect suspicious transactions fueled by various modeling technologies. Model vendors provide banks all aspects of transaction monitoring, investigation, and regulatory reporting. RMA’s December 2021 survey on vendor model validation and third-party risk management indicated that the most common vendors providing BSA and AML and fraud models include Abrigo, NICE Actimize, Experian, FISERV, Lexis Nexis, Mantas, NetReveal, SAS, and Verafin. 

What’s different?

BSA and AML models need to be validated contextually. In general, both internally developed and vendor-sourced models follow the same model validation rigor. However, unlike internally developed models, vendor models present specific challenges.

First, with vendor models, validators have limited access to critical information about model development and implementation processes. To tackle this issue, many validators assess each sub-model as an independent model. Each of these sub-models represents a different measure of the different risk strands the bank is subject to.

For instance, at RMA Model Validation Consortium, the validation team starts with an initial assessment of three BSA and AML sub-models: the customer risk rating (CRR), the transaction monitoring system (TMS) and the watchlist (WLS).

We arrange an initial walk-through of the model with the model owners at the client bank. We also discuss any unanswered questions regarding components in follow-up meetings with the vendor model representatives.

These two initial steps set the pace for the validation process and allow us to understand how well the bank knows its vendor model implementation process and how supportive the vendor has been throughout the process.

During the validation process, all validation activities are focused on the main goal: identifying and measuring any risks associated with the BSA and AML model.

We examine how well each sub-model performs, and how appropriate the calibration of the model is.  We also consider the relevance of the data used for the model calibration, considering the bank’s current risk exposure. Finally, we look at the current model governance and monitoring framework.

Avoiding the setbacks

Communicating with the model vendor can prevent many of these validation issues. We suggest banks request vendors regularly perform quality model validations on their models. This step can also supplement the risk identification and mitigation processes performed by the bank.

We often find some weaknesses in understanding of the vendor model by smaller banks. Although the expertise on model development remains with the vendor, banks are responsible for complying with all regulations related to a model usage including, in this case, a BSA and AML model.

Therefore, banks need to develop and oversee some of the key components of model risk management: fully understanding the model components, the flow of information in the tool, the role of data, model assumptions and limitations of the vendor model, and how the vendor model and the internal BSA and AML programs are connected.

Specifically for smaller institutions, we often find model monitoring activities are not as rigorous as they can be in the context of the model governance. BSA and AML models are considered high-risk models by regulators. Institutions need to ensure that there is an appropriate monitoring plan that constantly reviews and oversees the working models. This plan should identify model assumptions, track the performance of working models, as well as track the datasets that got fed into the models. The plan should also classify model risks associated with it and other portfolio-specific risk factors.

We also face limited access to model datasets when working with some of our client banks. This is more evident with smaller institutions.

The most urgent issue we face is getting access to internal datasets when performing model calibration. To resolve this challenge, most vendors provide proxy data. However, proxy data is most useful when the institution has an appropriate peer data selection process in place.

The usage of proxy data for vendor models needs to be very well thought out. If not, banks cannot select the most suitable proxy data for addressing their idiosyncratic risks, which can translate into an inadequate classification of risks under the BSA and AML program.

A lack of an appropriate model validation practice is another challenge that smaller institutions face. Model validations are key processes for any bank to support its regulatory compliance as well as internal risk management effort.

Make sure that a vendor you want to work with can perform the validations of their high-risk models. It is even better when they have all the quantitative and qualitative background and the experiences required in all aspects of developing and implementing BSA and AML models.

Have more model risk questions? Ask our Model Validation team. 

[i] https://www.federalreserve.gov/supervisionreg/topics/bsa.htm

[ii] https://www.occ.treas.gov/topics/supervision-and-examination/bsa/bsa-aml-examinations/index-bsa-aml-examinations.html

[iii] https://www.fdic.gov/resources/bankers/bank-secrecy-act/

[iv] https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm

[v] https://www.federalreserve.gov/supervisionreg/srletters/SR2108.htm