Can a credit analysis
model estimated on public firms generate valid inferences about default
probability for private firms? A discussion of the ratios of public and private
firms and their relationship to default.
Reprinted from the Commercial Lending Review, July 2000
Does a leverage ratio of 60% mean the same thing for Bob’s Computer Warehouse as it does for IBM? This question is relevant to any lender who evaluates credit exposures that span traditional market segments such as small business, middle market, and large corporate lending. It is also relevant to anyone interested in purchasing credit scores, as these are more often than not estimated on public firms, if not the more exclusive set of agency-rated public firms. In our opinion there are subtle yet significant differences between public and private firms that are best addressed through direct estimation and testing on private firms. This article discusses the similarities and differences between these two universes.
When rating portfolios of commercial credits, traditional subjective credit analysis is no longer feasible. These portfolios may contain thousands of obligors, simply too many to manually underwrite cost effectively. The only way to analyze these portfolios is by using quantitative methods. In many cases these techniques lead not only to more transparent, efficient, and comparable risk measures. Surprisingly, they are often more accurate as well.
In 1998, Moody’s created a new group, Risk Management Services, that focuses on quantitative tools and private firms. We are investing significant resources into building a private firm default modeling database because we have to analyze more and more private firms for the growing collateralized debt business. We also see private firm credit analysis as an area where we can leverage off our existing strengths to create benchmarks that will facilitate greater transparency for regulators, investors, and rating agencies.
The result of this effort is a default prediction model that can be applied to private firms. We estimated the latest version of this model on a private firm sample of over 1,500 defaulting firms for over 30,000 companies and 156,000 financial statements. (you may download the comprehensive technical document that describes the development and testing of this model in detail at http://www. Moodysqra.com).
Data drive default prediction models. Thus, the availability of financial data from Compustat, along with ratings and default information from S&P and Moody’s, has naturally led to these sources of information being the basis for many academic and private vendor models. One approach to creating a quantitative model is to use the rating (for example, Baa, A) as the dependent variable, such as in looking at the relative mean ratios by different rating. These methods include the ‘ordered probit’ model, and also translating the ratings into numbers (for example, Aaa=1, Aa1=2 … C=21) and then regressing the ratios onto these grade transformations. Another method is to estimate default directly for public firms, using a binary estimation method where the dependent variable is 1 if the firm defaults, 0 otherwise. Yet the outstanding question is whether a model estimated on these public firms generates valid inferences about default probability for private firms.
This article investigates this issue by examining the ratios of public and private firms and their relationship to default. While there are many similarities in how these ratios behave for public and private firms, there are also some key differences. These disparities highlight the importance of using private firm data to construct a model targeted at private firms.
The juxtaposition of two key ratios distributions illustrates an important point (Exhibit 1).
The ratios retained earnings/assets and interest coverage are representative of other ratios in that a large portion of their observations are outside the range of conventional analysis. It is common to have interest coverage rules using numbers like 0, 1, and 5, yet for many firms these numbers are negative or well above 10. Quantitative approaches have to be robust to these outliers if they are to be comprehensive in applicability. The mere fact that so many firms exist with these extreme ratios highlights that extreme values are not fatal. Further, linearly extrapolating a rule based on the 70% of cases that are “normal” will lead to wildly distorted inferences for the many firms that are not normal.
The ratio that shows a difference that is mainly in its distribution (as opposed to its average) between public and private firm ratios is retained earnings. Many public firms have highly negative retained earnings: 20% with less than 100% retained earnings/asset ratio. This is because many public firms take special charges, and in the case of large negative windfalls they can stay afloat due to the ability to dynamically refinance with the capital markets. Retained earnings are a powerful predictor of default for both private and public firms, yet an adjustment must be made for this vast disparity in the meaning of a significantly negative retained earnings measure.
Many of the inputs used in Moody’s private firm model show similar relations to default in the public and private datasets. Exhibit 2 shows frequency-of-default graphs for two ratios for public and private firms. The line represents the frequency of subsequent five-year defaults for firms by ratio, which was constructed by bucketing firms into 50 ordinally ranked groupings, observing their default rate over the next five years, and then smoothing the relationship to remove the “jerkiness” that appears in unsmoothed data. The public firm data are from 1980-1999; the private firm data is primarily from 1995-1999.
In general, the differences between the two universes are minor; the distinction between public and private is not significant for most ratios. The broad similarity between the two independent datasets suggests that what we observe are real relationships that transcend institutional boundaries, not simply anomalies of small samples. This gives comfort to the belief that ratios can help prediction, and these predictions can be calibrated and tested.
There are important exceptions, however. The first to note is size. Though public firms range in size from less than $1 million to $300 billion, with many firms having less than $20 million in assets, they are still much larger than the average middle-market firm. The median firm size in Moody’s private firm database is $2 million versus $100 million in Compustat.
The relation between size and default is not straightforward due to a selection bias in the Compustat data. Compustat does not include recently listed small-sized firms with spotty financial reporting. A small firm that haphazardly reports its financials or one that has existed for only a year or two is usually not listed. If such an unlisted firm defaults or goes bankrupt, it will never be registered. If it survives and adopts more rigorous accounting standards, it will then appear in Compustat with its prior financial statements “back filled.” Compustat, therefore, has what is known as a survivorship bias for small firms. Once a small firm is in Compustat, it has already survived perhaps the most crucial period of risk in a firm’s existence.
For the private firm database we encounter a similar issue but in the opposite direction. Many defaults in the database were added manually, and the manual selection process for defaults has a definite size bias. Larger firms are more apt to be remembered and kept. Using only manually collected defaults, one invariably finds that size and default are directly related: bigger size, higher default. This is not the data speaking, but the bias in the data. Fortunately most of our private firm default data come from an automated process that links financial statements with credit information, which mitigates much of the bias. We do not have sufficient numbers of defaults, however, to exclude the manually collected defaults. Therefore, we are careful to make some informed adjustments to the model, even though these adjustments are not justified solely by looking at the sample.
Exhibit 3 shows what relationship underlies the default probabilities of some of the ratios that display marked differences in behavior between public and private firms. As mentioned, for public firms size is positively correlated with default at the low end, and negatively correlated to default at the high end. One could reasonable infer no relationship based on an uncritical view of this data alone. Yet for the private data something different happens. Below $40 million there is a strong negative relation between default and size, after $40 million the relationship flattens, due mainly to those manually added defaults, which tended to be larger. It is our best estimate that size is monotonically related to default, though the relationship dampens as one increases in size.
As illustrated in Exhibit 4, public firms have very low retained earnings/assets, usually due to large extraordinary charges, while such occurrences are uncommon for private firms. This difference in the use of charges becomes reflected in the relationship between defaults and retained earnings for public and private firms. The monotonic relation between retained earnings/assets in the private dataset makes for a more robust and meaningful measure of risk in a model fit to private firms. A very low retained earnings level is usually a special case for public firms, and implies one should use a different lens when assessing these than from what one would use for private firms.
Lastly we show short-term debt/total debt (Exhibit 5). This variable leads by far to the most different behavior for the two types of firms in terms of its relationship to future default. As opposed to the public market, where high short-term debt/total debt implies sharply higher default rates at the upper end, for private firms the relationship is pure noise and there is no effect whatsoever. This is because most private firms rely primarily upon bank debt. As banks have vastly different regulatory capital requirements for 364-day facilities versus facilities that are above 364-days, there is a tendency to put what are ostensibly longer-term commitments into a shorter-term category even though their practical maturities are much longer. Debt maturity for private firms, therefore, is more related to institutional factors than risk, which contributes to the weak relation between this variable to default for private firms. One should be very wary of a model fitted to public firms being applied to private firms if it includes this variable. Such a model would err by assuming that short-term relative to long-term debt matters, when for private firms it does not.
Exhibits 6 and 7 illustrate the behavior of two ratios over time and their relation to Moody’s ratings. We can look to see if, in fact, the ratios that statistically drive risk, such as those used in Moody’s private firm model, show any trends over time by Moody’s rating category. Further, we can compare these trends to the unrated sample of public firms, as well as our own private firm database, to see what they imply about fitting a model to ratings and applying it to unrated or even private companies.
In Exhibit 6, we see that liabilities/assets show a predictable pattern between risk and ratings, as higher-rated companies have lower leverage ratios. For unrated public companies, inferences about their risk and ratings may seem valid as unrated public firms are generally in the Baa to Ba range, which is consistent with Moody’s estimate of the average default rate for unrated public companies. Yet unrated public firms have even lower leverage ratios than do Aaa companies. This highlights one of the difficulties of inferring from a public ratings model and applying these inferences to a model for private firms. Clearly very few unrated public firms deserve the Aaa rating; yet on this single dimension of leverage ratio perhaps most do.
The quick ratio was shown in Exhibit 2 to be strongly negatively related to default: the higher the quick ratio, the lower the default probability, for both public and private companies. Yet looking at ratings and quick ratios we see an opposite result: lower rated firms have significantly higher quick ratios than do their investment grade counterparts. Further, unlike unrated public firms, private firms have significantly lower quick ratios. This highlights the difficulty of creating a model based on ratings to estimate risk. A model based on ratings would map higher quick ratios into lower rating categories, even though from a default perspective it is negatively related to risk.
The mechanism at work here is that rated firms have varying degrees of access to outside credit, which affects their optimal holdings of current assets and liabilities, and higher rated firms with access to the commercial paper market do not need cash. A low quick ratio is an effect of a firm with low credit risk, not the cause of low credit risk. A model fit to ratings that uses a liquidity ratio should therefore be viewed with suspicion when applied to private firms (unreported charts for other liquidity ratios show a similar result).
Just as experiments performed on mice can be used to develop drugs for humans, models developed on public firms can help guide the development of models aimed at private firms. Yet just as a final study on humans is necessary for FDA approval, so too should a private firm model (or rule of thumb) apply testing and refinement based on private firm performance before it is adopted. Ratios related to size and liquidity are the most disparate in their implications for risk between the universe of private firms and that of their public or rated counterparts. The optimal weighting of ratios within a statistical model is not only subtle for private and public firms, but also, importantly, different. This difference suggests that in evaluating private firm credit models, you should be skeptical of any rules or models derived from relationships between the usual suspects: agency ratings and public firm financial statements.