Texas A&M University
R. Alton Gilbert
Challenges in Measuring the Risk Assumed by Individual Banks
Asian Regional Seminar on Financial Reform and Stability: Systemic Issues
- Administrative Staff College of India
- International Monetary Fund
March 29-30, 2001
My topic is the use of models for measuring the risk assumed by individual banks. I describe econometric models that have been developed by bank supervisors in the United States. The purpose of these models is to use bank accounting data to predict which banks will experience financial distress. This paper refers to several papers on early warning models and Internet links to recent papers. In addition, I discuss the conditions that are necessary for the use of econometric models to measure the risk assumed by individual banks, and possible approaches to bank surveillance if some of the conditions are not met.
My knowledge of bank supervision is limited primarily to the United States. The following BIS working paper indicates that among the G-10 countries only France and the United States use econometric models for bank surveillance.
Ranjana Sahajwala and Paul Van den Bergh. "Supervisory Risk Assessment and Early Warning Systems," Basle Committee on Banking Supervision Working Paper No. 4, Bank for International Settlements, December 2000.
I assume that the use of econometric models for surveillance is also rare in other nations.
Use of Econometric Models for Bank Surveillance in the United States
The Federal Reserve uses two econometric models for bank surveillance, which is the process of monitoring the condition of banks between on-site examinations. The SEER (System for Estimating Examination Ratings) risk rank model was estimated to predict bank failures. The coefficients of this model were estimated with observations on bank failures for the years 1985 through 1991. The model has not been re-estimated since 1991 because of the small number of bank failures in the United States since 1991. Federal Reserve staff rank banks by their probability of failure each quarter by plugging current accounting data into this model. The SEER rating model predicts the supervisory rating of each bank on its next examination. The following article provides information about these models.
Rebel A. Cole, Barbara G. Cornyn, and Jeffrey W. Gunther. "FIMS: A New Monitoring System for Banking Institutions," Federal Reserve Bulletin, January 1995, pp.1-15.
At the Federal Reserve Bank of St. Louis, we have been working on models to predict which banks will have their supervisory ratings downgraded to problem status in future periods. After the early 1990s the number of bank failures fell below a level appropriate for re-estimating the coefficients of a bank failure model each year. It is possible to estimate a model for predicting downgrades of supervisory ratings each year, however, because the number of banks downgraded to problem status each year has been much larger than the number of failures. The following article presents a model for predicting downgrades of supervisory ratings to problem status.
R. Alton Gilbert, Andrew P. Meyer, and Mark D. Vaughan. "The Role of Supervisory Screensand Econometric Models in Off-Site Surveillance," Federal Reserve Bank of St. Louis Review, November/December 1999, pp. 31-56.
Our article also demonstrates that predictions of the banks that will be downgraded to problem status each year derived from econometric models are more accurate than predictions based on individual financial ratios, which supervisors commonly call "screens."
Supervisors in our Bank use our model of downgrades in supervisory ratings to "scope" examinations, which is the process of identifying the major issues to address in an on-site examination before sending examiners to a bank. We use the coefficients of the model to predict the probability that a bank will be downgraded to problem status. In addition, we use the coefficients on the individual independent variables to determine the aspects of a bank’s operation that contribute most to its probability of being downgraded.
The Supervision and Regulation Department of the Federal Reserve Bank of Chicago has recently conducted research on use of a technique called "trait recognition" for bank surveillance. Trait recognition is a technique for examining the predictive power of combinations of financial ratios. This technique could be especially useful for surveillance in banking systems with small numbers of banks.
Julapa A. Jagtiani, James W. Kolari, Catherine M. Lemieux, and G. Hwan Shin. "Predicting Inadequate Capitalization: Early Warning System for Bank Supervision," Emerging Issues Series, Supervision and Regulation Department, Federal Reserve Bank of Chicago, September 2000 (S&R-2000-10R).
In addition, a working paper of the Office of the Comptroller of the Currency (the agency that supervises banks with national charters) has investigated the use of trait recognition for bank surveillance.
James Kolari, Dennis Glennon, Hwan Shin, and Michelle Caputo. "Predicting Large U.S. Commercial Bank Failures," Working Paper 2000-1, Comptroller of the Currency, January 2000.
The Office of the Comptroller of the Currency recently released the following document which includes a discussion of the use of early warning models by the staff of the Comptroller’s office."An Examiner’s Guide to ProblemBank Identification, Rehabilitation, and Resolution" January 2001.
Conditions that are Necessary for the Use of Econometric Models in Supervision
While the experience in the United States indicates that econometric models can be useful tools for bank surveillance, this result rests on some important features of our financial system. Unless your national system has these features, reliance on econometric models for surveillance could be useless or dangerous. The following is my list of conditions that are necessary for the use of econometric models for bank surveillance.
The causes for distress among banks in the future must be similar to the causes of distress in the past. Predictions of early warning models will have large errors if the influences that cause banks to experience financial distress in the future are different from the influences that caused financial distress in the past. The problems of agricultural banks in the United States in the 1980s illustrate a change in the causes of bank failures. Banks that specialized in lending to farmers had relatively low failure rates for several decades prior to the mid-1980s. The outlook for agricultural banks changed substantially in the early 1980s, however, as land prices and farm income began falling. Predictions of the financial problems of agricultural banks derived from an early warning model would have come too late to have helped supervisors identify the banks most vulnerable to failure. Failure rates of agricultural banks began to rise in the second half of 1984. Out-of-sample simulations of an early warning model would not have predicted the distress of agricultural banks until 1986 or 1987. Our paper cited above provides more information about the implications of this experience for the use of models and screens in bank surveillance.
Well-defined accounting principles and rigorous enforcement of penalties for violating the principles. Use of econometric models rests on the assumption that the data are accurate.
Frequent on-site examinations by competent examiners who are free of the political influence of bankers. Our experience in the United States indicates that on-site examinations are essential for validating the accuracy of the accounting data that bankers provide to their supervisors. The authors of the following working paper found that revisions to bank accounting data tended to be relatively large just after banks had been examined and their supervisory ratings downgraded.Jeffrey W. Gunther and Robert R. Moore, "Early Warning Models in Real Time," Financial Studies Working Paper No. 1-00, Federal Reserve Bank of Dallas, October 2000.
Consistent Regulatory Regime over Time. Estimation of early warning models involves predicting certain events, which may be bank failures, downgrades of supervisory ratings, or reductions of capital ratios to relatively low levels. The conditions for these events happening must remain essentially the same over the period when a model is estimated and simulated. If the supervisors change the rules for closing banks, change the conditions for downgrading supervisory ratings, or change the accounting rules that affect capital ratios, early warning models will not be useful for predicting which banks will have problems.
Supervision and regulation of savings and loan associations (S&Ls) in the United States during the 1980s illustrates a change in the regulatory regime. Increases in interest rates in the United States that began in the late 1970s reduced the market value of the assets of many S&Ls below the value of their liabilities. The supervisory agency for S&Ls permitted many bankrupt S&Ls to remain in operation. In addition, the supervisory agency changed the accounting rules to help the S&Ls avoid reporting negative net worth on their balance sheets. After this change of regime, an early warning model for savings and loan associations would have been useless as a means of predicting the problems of savings and loan associations during future periods.
Suppose the conditions in your national system are not consistent with one or more of these assumptions. How can you do bank surveillance? Your best option is to rely on a few screens that you think are the most reliable. Choice of screens requires your judgement, and you will not be able to validate your judgement with evidence from econometric models. Our research cited above indicates, however, that reliance on screens comes with a price. An econometric model would outperform screens if the assumptions I have listed are valid for your national system.