Sherry Thomas, Head of Catastrophe Management - Americas and James Burnett-Herkes, Senior Vice President
Model Suitability Analysis (MSA)® consists of a set of standard tests and protocols that benchmark the models against independent reference data for hazard, event frequencies, damage functions, losses and historical experience. These datasets are created by independent and credible third-party research institutions that have expertise in the respective subjects. Rather than reinventing the wheel and developing models that already exist, the MSA approach evaluates the scientific underpinnings of existing models to establish confidence where warranted, and to identify areas of uncertainty. Guy Carpenter aggregates this information into our MSA Knowledge Base, and establishes standard protocols that are efficient to execute and test all models using the same standard procedure to achieve homogeneity and fairness in the process.
The scientific appraisal aspect of MSA makes use of data from scientific and engineering authorities, which are defensible, reliable and independent of the model vendors. Such datasets allow a robust, transparent and independent evaluation of model performance relative to a balanced and consistent baseline.
Such review includes treatment of uncertainty inherent in all perils such as earthquake, hurricane and especially the severe convective perils. We also account for physically plausible but unobserved events in the evaluation, often with insight from our academic and expert partners.
Through an array of MSA tests, we help clients quantify how well a model represents the physical hazard being analyzed and the aspects of the model that directly impact loss volatility. The standard for model testing and evaluation through the MSA framework leads risk takers to more confident risk-informed decision.