As Karen Clark, founder of AIR and now an independent consultant, has said, “the black box started out as a useful tool for decision making, but then it grew to be very big and very powerful; the black box now makes the decisions (4).” While somewhat hyperbolic, there is also much truth to this aphorism. Models are tools, and a good tool user understands the strengths and limitations of the tool.
Dispelling the “black box” effect, though, will require a change in approach and attitude across the industry among the various stakeholders.
Modeling firms need to be more open about uncertainty. They are in an ideal position to lead the discussion. Unfortunately, competitive realities work against this. Like a statistical Gresham’s Law, bad (low) estimates of uncertainty threaten to drive out good (high) estimates; the first firm to reveal the true extent of uncertainty in its model risks suffering for it in the marketplace. Perhaps now is the time for an independent, blue ribbon panel – funded by a consortium of model users – to study the uncertainty issue in general and publish a guide to the amount of uncertainty that should apply to all state-of-the-art models.
Primary writers need to become better users of models and consumers of their output, knowing what to ask, when to question and how to interpret the results in the context of material uncertainties. Being in the best position to understand the details of their own portfolios, they should begin tailoring model output to better fit their way of doing business. They need to take control back from the black box. Underwriters should be informed by, not controlled by, model results. Portfolio optimization, applied blindly, must give way to a nuanced approach to portfolio management, robust to the underlying uncertainties.
As insurers’ appreciation of the magnitude of model uncertainty increases, they might find that non-indemnity (index-linked or parametric) risk transfer products appear more attractive than they had in the past. Basis risk will appear to be much less significant when put against a background of the uncertainty in the true level of coverage provided by indemnity products.
Reinsurers, in comparison, are already relatively good catastrophe model users/consumers. They (better) understand the uncertainties, use multiple commercial models and augment those with models of their own. They should not take advantage of their informational advantage over insurers to extract economic rents, however, by disingenuously pointing to a model change and acting surprised. Rather, they should consider what kinds of new products will create value as their customers’ understanding of uncertainty deepens. A robust trade in index-linked products, for example, would allow cat risk to be “sliced and diced” in a manner similar to that of other financial risks – facilitating more liquid trading and placing the ultimate risk (and uncertainty) where it belongs – in the portfolios of well diversified investors.
Rating agencies and solvency regulators, being somewhat behind in their understanding of the nuances of uncertainty and risk, need to become better educated. They need to understand that model results are indicative, not conclusive, and that changes in models should be treated as information, not structural changes to the way insurers and reinsurers do business. They must understand that model changes, as information, provide a noisy signal against the background din of uncertainty; that such information needs to be filtered, scrutinized and gradually absorbed. They should not encourage insurers to engage in meaningless and harmful optimization exercises to meet precise, but ultimately, fictional, targets.
Boards of directors, investors and stock analysts need to appreciate the comparisons between uncertainty in cat modeling results and other forms of uncertainty in the financial markets. Dr. Miller’s study is a good place to start, as it explicitly puts cat model uncertainty on a par with corporate bond default rate volatility. Also, the place of cat risk within the enterprise-wide spectrum of risk needs to be appreciated.
Insureds (homeowners and business owners) and other interested parties who do not routinely see cat model results need to understand that there is a considerable amount of uncertainty and that no one knows precisely how much risk is present in a given situation.
Brokers, finally, need to stay out in front to facilitate education, communication and fair dealing.
Conclusion
Uncertainty permeates the catastrophe modeling enterprise. When a cat model says “Your 100 year return period loss is $1,117,243,572,” what it really means is that your 100 year return period loss is about a billion dollars, but it could be 600 million dollars or maybe two billion dollars…or something like that.
While models have considerable uncertainty associated with them (even under a multi-model approach), they are still valuable tools. Model outputs must be assessed against this background of uncertainty, however, and blind reliance on model output should become a historical footnote. Portfolio management needs to understand the details of model uncertainties, and board level risk assessment needs a rigorous ERM framework for guidance.
All the players in the property-casualty (re)insurance industry need to understand, discuss and act upon the reality of cat model uncertainty – as well as the certainty of continued changes to the models themselves. Until these models can measure risk with perfect precision – a Utopian ideal unlikely to be achieved – they will continue to evolve and at least bring it into clearer focus.
How Guy Carpenter Can Help
Clearly, there are large changes occurring in major cat models – and clearly, these changes are going to continue to occur on a regular basis. To keep up with the implications of change takes a dedicated fulltime staff, not just to ensure rational use of models, but to stay abreast of all the possible knock-on effects, for example, primary pricing, reinsurance pricing or interaction with regulators/rating agencies.
Guy Carpenter is committed to offering industry-leading analytical and advisory services to our clients based on superior analytical skills, theoretical understanding and practical know how. Our dedicated professional staff works daily with multiple cat models and modeling firms as well as Guy Carpenter’s own proprietary cat models and analysis tools. In fact, we have more experience running models than many of the modeling firms themselves.
Guy Carpenter advises clients to consider a multi-model approach. In particular, we can show our clients how to combine the results from multiple models in a scientifically credible fashion, taking into account the individual characteristics of the client’s portfolio of exposures and historical loss experience. Additionally, we can advise clients about the relative strengths and weaknesses of models in their various aspects, for example, perils or geography, and how “model blending” can combine the best from multiple models.
In particular, Guy Carpenter provides significant access to, and frequent technical discussions with, cat model vendors. Since the release of the RMS model version 11 in early 2011, Guy Carpenter has performed extensive analyses to help clients understand the impact of model changes on their portfolios.
MetaRisk®
In today’s challenging environment, your ability to clearly see an evolving landscape, assess options and make proactive risk and capital management decisions will set you apart from competitors. Guy Carpenter’s MetaRisk®, the industry’s most transparent risk and capital decision tool, delivers unprecedented clarity. It gives you the power to see, understand and interact with the drivers of risk, so you can make business-critical decisions with confidence.
MetaRisk puts transparent and auditable information into your hands so you can validate your risk and capital management position with all of your constituents. MetaRisk provides a wide range of standard financial metrics as outputs so you can clearly understand the impact to your bottom line. Furthermore, its integrated, comprehensive capabilities can assist you in meeting the rigorous demands of ERM, Solvency II and other regulatory requirements.
MetaRisk is an ideal tool to implement model blending.
i-aXs®
Quickly and easily managing vast amounts of data is critical to your success. Guy Carpenter’s i-aXs® provides a full suite of tools to help you translate your data instantly, allowing for faster and better informed decisions. Delivered via an easy-to-use homepage, the award-winning platform integrates sophisticated data analyses systems, cutting-edge spatial technology and satellite imagery. In a few quick keystrokes, you can view, graph and map your data. i-aXs enables you to assess one portfolio, combine multiple portfolios or drill down into the data to individual locations. And with access via the Web 24 hours a day, seven days a week, your data can support you whenever, wherever you want. With the latest technology and a unique set of tools, i-aXs offers unprecedented features.
Portfolio management, as applied to exposure to loss from natural and man-made catastrophes, is a mission-critical pillar in an ERM framework. In addition to managing and controlling catastrophe losses and producing stable results over time, adequate pricing for property exposures (including cat and non-cat losses, plus the cost of reinsurance, expenses and capital) plays a key role in demonstrating a firm’s risk management acumen and strategy.
Guy Carpenter’s portfolio management tools – Policy Ranking, Gradient and Reinsurance Cost Allocation – can operate on modified or blended model results. A Guy Carpenter team can show you how to combine the power of MetaRisk and i-aXs tools into a robust portfolio management strategy.
Note:
4 Clark, Karen (2008) Thinking Outside the Black BoxTM: Catastrophe Risk Management Best Practices, 2008 Southeastern Regulators Association Conference, Orlando, FL, Oct 19-21.