In the economic environment created by the COVID-19 pandemic, many models on which financial institutions rely for their business decisions became inadequate. The extraordinary economic conditions exacerbated preexisting stresses in model risk management (MRM). Facing a critical challenge, a few leading institutions, with others following suit, have begun to rethink their model landscapes and the model life cycle. As we discussed recently, their considerations have revealed a new S-curve in model risk management.
In the past year, McKinsey provided a number of forums for model risk managers from financial institutions around the world. These professionals shared their views on challenges and emerging themes at roundtables and in our global MRM survey. More than 150 model risk managers from nearly 100 institutions participated in the survey. The results showed three areas of focus: the impact of COVID-19 on MRM, the evolutionary changes in MRM, and the challenges posed by models using artificial intelligence (AI) and machine learning (ML).
The impact of the COVID-19 pandemic on model risk management
The COVID-19 pandemic has affected the performance of models globally, including traditional financial-risk models, such as those for credit risk, as well as models for nonfinancial risk, such as fraud management. Banks necessarily responded with short-term measures. The responses included more frequent monitoring to identify models at risk, compensating controls such as model overlays, and substitutions using alternative existing models. Some of these short-term measures (such as overlays) lacked adequate controls, so appropriate governance had to be developed.
Leading institutions are also undertaking longer-term solutions, upgrading their MRM functions to adapt to the new conditions. Their objective is to move MRM to a new level, defined by a meaningful collaboration between the first and second lines of defense. For that to happen, institutions will have to embed MRM culture throughout the model life cycle. That means changing not only processes and procedures but also the ways individuals involved in the model life cycle think and act.
Heightened model risk and emerging solutions
For financial institutions, the pandemic’s effects increased model risk in a wide range of use cases. Models that rely on macroeconomic variables or customer behavior were heavily affected. Specifically, models for predicting creditworthiness, as well as for stress-testing and provisioning, were all severely tested by the pandemic-triggered economic fallout.
In terms of model impact, our survey revealed that for most banks (more than 80 percent), the most heavily affected models were those for credit risk and stress-testing. These models rely on static historical data, and the pandemic created discontinuities and problems of data reliability. To address the challenges, institutions in different regions are taking different approaches. In Asian countries, especially China, banks are already recalibrating or redeveloping their models. In North America and Europe, model remediation is taking the form of interim overlays (such as expert judgment) as the search for more systematic approaches proceeds apace.
From expert discussions on the challenges created by quick remediation, some best-practice solutions emerged. The MRM function must establish governance of overlays covering business-as-usual models and regulatory models. Transparency of all overlays is vital, as banks explore more systematic and responsive approaches to address overlays in every segment (consumer, small and medium-size enterprise, corporate). That is needed as the COVID-19 pandemic and its effects are stretching across a longer time horizon than was initially anticipated.
Overlays cannot always meet the challenges effectively. Institutions should consider prioritizing models for redevelopment, where feasible and effective. The first and second lines of defense must collaborate closely to identify the affected models and guide redevelopment or recalibration.
Participants in our MRM discussions noted that a “crisis tool kit” could help institutions that are relying on second-line controls and model monitoring. These crisis procedures and protocols would incorporate lessons from the present crisis in order to provide a contingency plan to deal with the next one. Some of the signal findings from our MRM survey are listed in the sidebar.
The evolution of model risk management
The crisis has highlighted the value of MRM and raised the function’s significance as a strategic-risk partner. MRM maturity varies by region, in part due to different regulatory guidelines. Nonetheless, improving validation effectiveness and operational efficiency are universal priorities. Our survey revealed that the number of models requiring validation and risk reviews is growing, and the scope of MRM is also rapidly expanding—into models for automatic decision making, for example.
Within financial institutions in every region, MRM functions are evolving faster than ever, primarily because models are proliferating in number and scope. The survey revealed that most institutions are enhancing their MRM frameworks as a priority. Respondents agree that this is most critical in regions where regulatory pressure is higher.
Challenges
Banks face cost and capacity pressures as they strengthen frameworks and expand model inventories. Validation backlogs and delays mount as existing validation capacity fails to cover expanding demand. Inventory is increasing as new models are developed outside traditional areas of financial risk. The rapid development of AI is increasing model complexity and adding to the backlog.
The quality of validation can consequently suffer unless the bank brings in external support. To manage the model-validation budget, leading banks have industrialized validation, using lean fundamentals and automated processes. Models are prioritized for validation based on key factors such as their importance in business decisions and materiality of the model exposure. Validation intensity is customized by model tiers to improve speed and efficiency. Likewise, model tiers are used to define the resource strategy and governance approach.
The use of model tiers to improve efficiency varies by region. In Asia and Latin America, where MRM functions are still maturing, about half our surveyed banks report using tiers in their model inventory. In Europe, tiering is prevalent, but most banks do not use it to its full effect, deploying it to determine validation frequency but not the depth of validation. In the United States, most large banks refined their framework by including a fourth tier in their model classification. The additional tier is essential for the impact of tiering to be effective, since the number of models is steadily increasing.
Solutions
The next level of maturity in the MRM journey is defined by more advanced MRM capabilities, which go beyond the validation-centered approach. The emphasis shifts from a technical model review to a risk manager’s view that assesses the risks beyond model methodology. The entire portfolio of models is managed, including extended inventories beyond credit- and market-risk models, encompassing also nonfinancial-risk and business models. Reporting thereby becomes meaningful, as senior managers get an exhaustive view on model risk beyond the technicalities—one with a real risk perspective.
Learn more about Risk Dynamics?
Institutions can take targeted actions to realize their MRM objectives. The MRM function should define priority actions to improve governance, frameworks, model scope, and standards for model development and validation as a foundational phase for an efficient operating model. Improved validation is an obvious top priority, especially in North America and Europe, where MRM is more mature. More than half the survey participants identified automation as the most important approach for improving validation efficiency given the current requirements and scope in these regions. Validation remains a priority, since it ensures that models are of high quality and do not generate undue risk. The key challenge is to balance quality and efficiency in model validation, in recognition of current cost pressures.
Efficiency can be further improved by a review of the model landscape. Where banks can simplify the overall landscape, they will also ease the validation workload. The number of models and their use cases are rising rapidly, including use cases outside areas of risk. Nearly half of North American and European survey participants reported that a better understanding of model interdependence is an important precondition for prioritizing models and streamlining MRM activities.
Artificial-intelligence and machine-learning challenges
Institutions are increasingly using models based on artificial intelligence and machine learning. AI and ML models amplify model risk because of their complexity and comparative lack of transparency. Complicating issues include designer bias, which, given the nature of these models, is difficult to detect; interpretability, meaning the ease or difficulty of predicting what a model will do; and explainability, defined as the degree to which the workings of an AI or ML system can be understood in nontechnical terms.
Talent is lacking
To keep pace with these AI and ML developments, MRM must shape standards and perform end-to-end management for the new models. Most MRM functions do not have comprehensive standards tailored for AI and ML. These are needed to address specific challenges, including bias detection, ethical questions, and explainability. Further vulnerabilities are caused by a lack of appropriate AI and ML tools and infrastructure. The steepest challenge, however, is in the area of knowledge. Most MRM functions are short of AI and ML talent. Model submissions are often incomplete, furthermore, as many owners of the new models neither adequately understand the responsibilities of a model owner nor have a sufficient grasp of model risk. Early signs are that institutions will face increased regulatory scrutiny of AI and ML models as they adopt use cases at scale.
Our survey revealed that validation of AI and ML models is in a very early stage in all regions, though Asian institutions are more advanced in model development. Among Asian banks surveyed, 90 percent plan to develop more AI and ML models over the next two years. In addition, the accelerating pace of digital transformations, partly brought on by the economic crisis caused by the pandemic, is causing demand for these models to increase. Yet less than 20 percent of surveyed banks said that they were ready for this demand. Many cited a lack of AI–ML talent as their most glaring shortcoming in this regard.
Recruiting expertise and building capabilities
To help their institutions adjust to this fast-changing environment, MRM functions must lead the campaign to attract sufficient expertise in advanced analytics. Once the needed talent is in place, MRM functions can keep pace with AI–ML development—establishing needed training programs, selecting the right tools and infrastructure, and developing appropriate standards. These capabilities are urgently needed to support use cases for AI–ML models, which are quickly shifting from experimental pilots to extension at scale. Surveyed banks noted that the first line is incorporating techniques that may be insufficiently rigorous for these more complex models. Active management by the MRM function of the tiering approach to model development and validation is thus clearly needed.
Regulators are now giving attention to MRM and model governance in the application of analytics to digital- and internet-lending use cases. In January 2020, for example, the European Banking Authority issued guidance for banks on improving controls in their implementation of advanced analytics. China’s Banking and Insurance Regulatory Commission introduced specific requirements in July 2020 to expedite governance for AI–ML models. AI has also increased the importance of data management for MRM frameworks. AI–ML data requirements are significant: compared with traditional models, AI–ML models consume far greater volumes of data, including from third-party sources. The complexity of the data and the number and variety of use cases are also greater. Institutions need to be able to apply rigorous data-management frameworks and MRM, with clearly defined model-related data-governance and ownership structures.
In this next-generation MRM environment, the MRM function must apply its robust risk-management practices across the model life cycle. By ensuring that effective oversight is built into processes, the MRM function also fosters closer collaboration with the first line in managing model risk. The result is a sustainable operating model.
The importance of MRM was already growing before the COVID-19 crisis. In response to rising levels of risk and the need for more sophisticated modeling, financial institutions began to develop AI–ML models for financial and nonfinancial risks alike. The crisis has accelerated digital transformations in the financial sector, which has been an important driver of the new generation of models.
This, then, is the right moment to transform MRM. The function’s strategic importance has increased. Across the organization, the scope of models is expanding; many of the new models are designed around advanced analytics. The level of MRM work is rising commensurately, calling for greater MRM efficiency. Financial institutions need a less validation-centric function, one that can strategically prioritize the redevelopment and adjustment of models. A more comprehensive MRM approach, beyond validation, will help ensure a model life cycle better suited to AI and ML models.
