IRM on Internal Model Validation - Red Card or Green Card?

Cyclic Validation - Quelle horreur...

Just back from Paris, where I spent a weekend queueing behind selfie-taking tourists before taking out a second mortgage to buy bottled water. A beautiful place, though I found Depardieu was much quieter off-screen...

Onto the topic in hand, the PRA were pretty vicious back in the day on Validation efforts in their infancy, with Julian Adams lambasting both progress ("significantly behind") and validation scope ("narrow"). Given that the Solvency II sabbatical which bridged half of 2012 and all of 2013 gave firms time to catch up and widen, you might think that those with internal model ambitions would be pretty tidy by now. The PRA have even told firms how they believe "good" model application paperwork to look, carving out for themselves and the Validators of the world an easy-to-read "model reviewer" level of detail (p1).

In those salad days, Internal Model Validation felt to me like it would be the chernozem of the nascent Risk Management profession in insurers; a skill set that a quant or a non-quant could acquire, apply, and ultimately ease through the promotional path within insurance entities, given the depth and breadth of technical and strategic information the process challenges...

...but the moves never came. Despite the actuarial world themselves happily disassembling the complexities of quantitative modelling into easy-to-digest IM Validation themes, the non-quant world has waited patiently to see if anything of substance would emerge from one of its representative bodies.

And this week it arrived! The Institute of Risk Management has delivered, as part of its Internal Model Industry Forum (IMIF), a white paper on the validation cycle.

The IRM have been active in this area prior to the formation of the IMIF. I have covered an ERM in Insurance event at the start of 2014 here, while this more volumous slide pack featuring a number of the Billy and Betty Big Biscuits of the field emerged from summer of last year, when the IMIF seemed to come to fruition. This white paper itself appears to move along the concepts and ideas inside an IRM slide deck from last Christmas.

Given that the IRM is not-for-profit, there is always a likelihood that sponsors will unduly influence the products (indeed the IRM Chair notes in this that they rely on "enlightened industry support" to knock these documents out).

Sadly in this case, the sponsors include Three of the "Big 4" (with the fourth on the IMIF steering committee) , leaving the document dripping with consultancy hallmarks rather than pragmatic solutions to execute the tasks in hand.

That view is reinforced somewhat by this follow-on presentation to the IMIF from last week by this white paper's workstream lead and supporting consultant - one selected industry comment on slide 8 (presumably from a chocolate bar shortly before it ate itself) reads, "validators should really be experienced modellers"!

A few general points jump out of the white paper;

• That a firm's IM is "...at the heart of risk and capital evaluation" - I thought it was supposed to "inform" this evaluation, not dominate it (slide 3 here, as well as Julian Adams's speech from a couple of years ago [p4]).
• Is the insurance industry "...increasingly reliant on sophisticated models" - maybe in terms of AUM/Market Cap, but given the UK IMAP queue is down to approximately 40 firms out of over 400 (p4), and that number has steadily reduced over the last 3 years, feels a touch disingenuous. I've no doubt the firms represented on the Steering Group are "...increasingly reliant" though
• The document claims to set out "best practice principles" - not sure if "practice" and "principle" share the same bed, but that aside, would anyone find it remotely acceptable to have the consultancy world fund a document which details "best practice" on IM Validation?

And a few stand out elements from the proposed Validation Cycle, which is heavily influenced by EIOPA's guidelines:

• "Best practice now requires firms to demonstrate, with evidence, that the cycle...[is] being actively and effectively carried out" - how can best practice "require" anything from anyone?
• "...resulting best practice that is emerging" (p4)  - how is any practice considered "best" at this stage of proceedings, when we are literally practising! Against what criteria?
• References to "model risk impact assessment" and the "model risk assessment process" (p5) seem to come from nowhere. Alluding to something formal, but not very clear
• Lot of coverage of "triggers" of IM Validation, which feels like a fishing expedition for the paper sponsors, rather than direct address of L2 Art 241 - the number of areas of "change" to consider as IM Validation triggers covers pretty much any change, anywhere, both inside and outside of an insurer (p8)! Most would also be ad-hoc ORSA triggers in my experience, so this potentially sets up insurers for a bucketload of work every time they hear a pin drop.
• Formulaic and periodic IM Validation a "needless cost"? Surely periodic validation, no matter how badly executed, is compulsory (L1 Art 125)?
• The Trigger Impact Assessment stage (p10) is barely legible - "The trigger impact assessment against model risk appetite stage" - and terminologically it is all well above legislative requirements.
• "Unexpected triggers" (p12) get a mention. Again, not making sense to me - you either know your triggers or not.
• "Model validation is complex" and "less than black and white" (p16) - certainly is if you try and follow this process! A focus on plain questions and less quant can only help the models non-expert users (slide 7).
• If the validation cycle, processes and execution are "continuously evolving" (p18), are they reliable? Feels difficult to meet L2 Art 241.3, at least from a planning and execution perspective, if the process is constantly being tinkered with 
• "Developing a communications strategy" (p20) as part of the validation scoping and planning stage feels terribly over-elaborate.
• "Robust planning" expected to be common (p22), which doesn't necessarily marry up with the expectation of dynamic rather than cyclic validation in future (p10)

I think it is right to take the hump to a certain extent here. The PRA have been cunningly silent on capital add-ons to date, but given the implication that they will not be applied and renewed ICG-style (slide 13), there is likely to be many more less monied Partial IM applicants to follow over the next couple of years. Having the most influential consultancy firms decide on what is "best" in the validation world (and for it to have this many bells, whistles and legislative off-roads) feels like setting those firms up for either a fall, or another bill.

The PRA actually delivered something with much less padding to the IRM back at the end of 2013, so I'm struggling to see why that has justifiably been turbo-charged. Given they have three of their finest involved with the IMIF, but are continuing to be directly vocal on this topic (as recently as March 2015), it sends a worrying message to the capital add-on brigade that the IMAP early birds will be setting disproportionately high bars for 2017 and beyond when they deliver their PIMs.

Ultimately, I was disappointed by the publication, which reads more like a flannel manual, and is certainly not the kind of Risk Profession contribution that the topic so badly needs if the PRA's dreams of Board's "directing" and "owning" the IM valdiation process (slide 9) are ever going to come true. The 200 page novella world of Validation Reporting feels closer than ever...

Towers Watson's Global ERM Survey - Knowing ERM, Knowing You...

A couple of treats from two of the powerhouses of the 'writing things down' industry on the practical use of ERM to drive decision making, rather than simply accompany it.

Towers Watson are targeting the Solvency II audience (at least on this side of the Atlantic) with a timely release of the results of their 8th Biennial Global ERM Survey. I say the results, as there is no sign of the full survey itself - any closer to their chest, it would be an areola's backpack...

As ever, these kinds of publications oscillate between flannel and insight, so while I cover those below, feel free to read the infographic and call it quits!

General observations from the main press release include;

• Three-quarters of (the almost 400) respondents say they are viewed as "important strategic partners" by the Board and Executive - I'm less inclined to see that as a mark of superiority, given that risk functions in some firms won't have the ambition or aptitude to achieve that status
• Implication that some respondents do not have a risk appetite framework in place - very worrying, unless this is just bad wording.
• Some firms said to be only "...using ERM for regulatory compliance". It may depend on jurisdiction, but I'm not inclined to agree that is even possible.
• The "ultimate vision" for a firm's ERM capabilities is referred to, which is a brow furrer, even conceptually. TW seem to bundle up risk culture, risk monitoring and risk tolerance into the "Vision" bucket, in case that term takes your fancy.
• The expression "very strategic approach" appears in print for the first time!

Getting Value from ERM?
- "Kiss my Face"

From the more elaborate Q&A document, we find the main granular material which TW were prepared to publish. Fortunately for readers this side of the Atlantic, the EMEA Director Mike Wilkinson holds sway over much of that conversation, including his tale of the firm who recently had an ERM/Business Strategy-inspired "Aha" moment.

That session contains a fair bit of contention, such as;

• Asking the questions "What's the purpose of risk management" or indeed the "purpose of your ERM Program" in the Q&A - if these had been directed to the respondents themselves, it would have contextualised a number of the seemingly negative responses i.e. If the purpose of your ERM Program is "don't get shut down", you are probably less bothered about being a "strategic partner"!
• That the business should "...challenge the risk group to create reports that help them make decisions" - Excel Jockey is hardly the work of a strategic partner...
• In a similar vein, that insurers are "drowning in data, drowning in metrics" - hardly a new phenomenon, and doesn't give any credit to the critical faculties of employees to filter what they do have.
• "...many [internal capital] models have matured" - a sharp intake of breath can be heard down at Moorgate!
• That "...an ERM Program can't properly be assessed until it has been in place for a while" - pretty sure the S&P crowd wouldn't hold off assessing you while you "embed"

Mike in particular does manage to keep a good focus throughout the Q&A on maximising trade-offs between risk and return being the big differentiator between Risk functions who are capable of influencing strategic decision making, and those who are perhaps more likely to be tabling red-amber-green reports tracking the outcomes of decisions which have already been made.

Other strong points include;

• In the context of Risk Tolerance, how to cater for the discretion required by an insurer's asset managers in handling investment portfolios.
• Touches on a couple of pieces which stood out in the CRO Forum's Risk Appetite publication last month, namely around the increasing number of measures being used to run businesses other than capital, allowance of movement within risk tolerance levels, and whether firms have effectively articulated their organisation-wide Risk Appetite and Risk Tolerance limits down into its subsidiaries/departments.

One aspect which gnawed at me throughout this reading is the constant referrals to "ERM Programs" - I don't think I am bathing in semantics to suggest that Programs normally start and end, whilst ERM would surely constitute a Framework. You might choose to redecorate the Framework periodically with a Program (Solvency II a prime example), but you wouldn't expect a Program to "mature" or "evolve", you expect it to conclude!

Nitpicking?

Central Bank of Ireland speeches - "and there's more"...

Solvency II-ready?
"It's the way I tell them"...

I rejoiced on Friday at the sight of more speech material emerging from the Central Bank of Ireland directorate, if only due to the Frank Carson* gag I could wheel out due to the volume of their recent speech-giving...

As an industry we should always be happy to hear the regulator on lead vocals, so I gave the pair of speeches released a once-over to see what Irish concerns have justified the recent bounty of public addresses.

Deputy Governor Cyril Roux was very targeted in his speech, delivered to PwC's Annual CEO Dinner. It apparently gave him "great pleasure" to be in PwC's offices, which presumably means they weren't on the meter...

Some of the statistics and comments served to highlight that Ireland is something of a special case in the context of Solvency II, in that two-thirds of Irish gross premiums are to cover 'foreign risks', and that many insurers under their auspices will not have proximity to or oversight of much of their distribution network.

A few messages jumped out from the rest of the speech;

• A lot of positive messages had a caveat implicitly wrapped with them - "...we are in the main satisfied with your engagement with the Central Bank"; "On the whole international firms generally file returns on time..."; "I also commend your general adherence to our Corporate Governance Code..."
• Goes as far as using the IMF's recent review findings to tell firms to stop poaching regulatory staff while simultaneously complaining about turnaround time!
• Nice point about keeping focused on current risks through the PRISM framework, rather than drifting into Solvency II mode before 2016.
• Having recently been complimented by Sr. Bernadino on Ireland's reserving governance (p12), he reinforced that assumptions pertaining to reserves are expected to be "critically debated".
• On ORSA, that the CBoI "...expects to see Boards actively directing the use of risk management tools...such s stress or scenario testing"
• On Internal Modelling, he not only expects Boards to "...have sufficient knowledge and skill to challenge the model outputs", but adds that they "...like to see a Board direct the modellers in their firms to run specific stresses and scenarios prior to an item being discussed at the Board" - a big advance on previous murmurings on use test from supervisory bodies.
• Pulls up firms who are seemingly not tailoring their model's parameters for the Irish-specific business.
• Similarly a message of insisting that cross-border distributors tailor Group-driven materials and processes for the Irish market such as "...group policies and output, such as the ORSA, and internal model...".
• A cute but important distinction that "embedding" Solvency II, rather than complying with it on paper, is still going to take considerable effort.

Sylvia Cronin's speech (well, the Solvency II aspect of it) stayed along the same lines as she pursued at the Industry event in late April, where she was harsh on a number of specific elements in preparatory phase ORSA Reports which had been observed.

In a section of the speech covering "challenges to be overcome", a number of pieces of insistent ORSA direction are given, for example;

• "Your Board must use the ORSA to more fully align business strategy and capital"
• "You also need to use it as a lever to discharge your core responsibility not to take on risks and exposures which the capital base does not support".
•  "...there is a lot of work yet to do by firms to get this element of the new regime embedded to the extent we required" - I add here that, given they will have only reviewed 2014's preparatory phase ORSA Reports and Processes, is this not a given, particularly after CBoI sponsored a template-filling approach for the smaller firms?

On the wider world, the speech covers;

• That Solvency II sets out "clear standards and expectations around your internal control and risk management" - agree on the latter, but the former?
• Believes that the "scope for subjective judgement" may open up regulatory arbitrage opportunities, and that "a number of iterations" will be required before EU-wide consistency is achieved, in a sly dig at, errrr, everyone in mainland Europe! 
• Similarly, the volume of cross border business HQd in Dublin poses a problem due to the geographical boundary of CBoI's "prudential remit"
• Reinforces the message fro April that Pillar 3 readiness is a growing concern
• A large suite of views on Conduct Risk, where "culture" and "conduct" are hogtied together as the grimmest twins since DeVito and Schwarzenegger - that message won't be changing in a hurry, so I strongly recommend your work in that area caters to the supervisor's tastes.

Useful insight from what appears to be a supervisor with their sleeves rolled-up - keep up the good work.

* PS I know the connection is tenuous as he's a Belfast man, but give me a chance!

Solvency II Industry Forum in Ireland - Clint and Chocolate

Having treated the speech by EIOPA's Chair as the main course a few days ago, I'd better take a look at the hors d'ouevres and pudding. At the CBoI's industry event in late April, a couple of their biggest hitters delivered speeches of their own, with a similar tone to the PRA's effort in October (i.e. a considered yet firm rollicking).


Solvency II Marathon
- no Snickering...

Sylvia Cronin's speech, in which she referred to Solvency II as the "Marathon of Marathons", shone some light on how Ireland plc has dealt with ORSA during the preparatory phase. If you remember, the CBoI kindly provided ORSA templates for the entities lower down the PRISM spectrum, which I felt at the time was fully justifiable, and a tactic other NSAs should have adopted.

It wouldn't appear that the industry has performed exceptionally in their 2014 efforts, with Ms Cronin drawing attention to the following flaws in execution

• Variances in ORSA Report content between firms in similar sectors  "cause for concern" - a slightly worrying comment, given that some firms will be over-elaborating with content with the help of their friendly consultancy firm.
• Boards were aggressively called out on the "ownership" front. If your Boards have been ambivalent to-date with regards to participation in the ORSA process, they will seemingly get a hard time from now on.
• ORSA Process "...as important as the document itself" - music to my ears, and still a surprisingly difficult concept to convey
• The "so called use test" is referenced around ORSA, and not in the context of internal model applicants - no idea what that is about, so will assume it was a clumsy turn of phrase, rather than a new element of legislation
• Stress testing seen to be "too benign", with firms ignoring key risks and hard-to-quantify risks. Given that the UK firms received similar feedback on their 2014 ORSA stress testing efforts, it doesn't appear to be a country-specific failing.
• 'Local' (i.e. Irish) ORSAs which filter into wider Group ORSAs have not been adapted to fit the business model of the Irish entity. Good issue to pull firms up on, if they are relying Head Office to provide them with process and reports in template format 
• Business plan and forward looking time horizons not considered plausible, possibly a by-product of the lack of Board involvement in the ORSA Process

On the Pillar 3 front, she noted the following

• Pillar 3 is now a "cause for concern" - decisions by firms on architecture and expenditure "now overdue".
• Only half of High or Medium PRISM firms have participated in external user testing on Pillar 3. This still feels proportionally more than the PRA have tested out, though it is still viewed as a negative by the Bank.

"Problematic Applicant"?

Cyril Roux on the other hand wanted to reflect on "the good the bad and the ugly of Solvency II" in his address. While most executives paying for Solvency II programmes in EU insurance entities would suggest that "A Few Dollars More" is a more apt Clint Eastwood hook, he took a more macro view during which he;

• Patted the Irish regulator for its "good work" putting the house back in order after the laissez-faire approach at the start of this century. 
• Also reiterated that Pillar 3 preparations are light in the country, urging firms to "...devote the resources necessary"
• Dwelled heavily on the subjects of investment risk appetite and prudent person principle
• Pointed at "problematic applicants" who can now seek approvals to do business in the EU regardless of the supervisor's thoughts about their business models
• Called Solvency II's quantitative requirements "ill conceived" and "overboard" and "clearly too complex"
• Commented that SCR is "unlikely, unreliable tool to manage capital" - "...certainly the supervisor will not be using it for more than its worth".
• Gave short shrift to the abolition of prudential reserving under Solvency II, and indeed Bernadino referred glowingly to the Irish approach to reserving governance (presumably to counteract this grievance?) in his keynote speech.

A lot to be taken from these speeches if you have any dealings with Irish insurers, and with the clock ticking, I can't imagine there will be better steers than these during the preparatory phase.

EIOPA Chair's speech in Ireland - Nothing Compares to U2...

The huff has concluded it would appear – EIOPA’s gaffer has returned to the speaking circuit after a few weeks where the institution’s communications had been reduced to Post-It notes on the fridge after the EU butchered their budget.

Bernadino - dodgy stand-up

 

A recent Solvency II industry event was held in sunny Dublin, with Sr. Bernadino providing the keynote address. He chose to start and conclude with U2 jokes, which were as lame as a constipated flamingo (I would have thought Ben Folds Five or Lulu would be top of his playlist right now…)

That aside, the meat in this musical sandwich was pertinent to the whole industry, so I've picked out a few highlights;

General

• Wants to outline EIOPA’s move “from regulation to supervision” (p2), though goes on to say that “…EIOPA does not replace NSAs. The responsibility of the day-to-day supervision…rests with the NSAs” (p10). I think this subject warrants clarification from NSAs and EIOPA in concert, given we can see how having a remit-less overseer is putting the PRA on the back foot
• Acknowledges that the result of the mathematical squabbles is “…perhaps a too complex SCR formulation” (p3)

ORSA

• ORSA considered to be “…best practice at international level” (p4) – not sure if he means ORSA in general IAIS terms, or the ORSA concept he has curated within the EU!
• While he drops some of the usual ORSA bluster in about it being a “game changer”, he beefs up on the Board’s obligations, citing their "fundamental role to play", and stating in particular that "they need to set, communicate and enforce a risk culture".
• Of particular significance to his Irish audience was his emphasis on risk culture as "..an appropriate balance with the natural sales driven culture". This is perhaps the first instance where I have seen insurers' distribution arms formally considered by a supervisory body to be the enemy of 'risk culture', and for EIOPA's chair to choose Ireland, a country which has for years marketed itself as a cross-border sales centre on the right side of General Good provisions, is shown at the end speech to be a pre-emptive strike.

Internal Models

• Like his associates at the PRA, he notes that the use of internal models on the banking side "has been subject to increasing scepticism" in justifying the rigour of the Solvency II approach to modelling
• Worried that models will become a "capital optimization tool" - the PRA's NED briefing (slide 8) paints a similar picture if you read between the lines regarding missing risks and experience not reflected in parameter setting.

EIOPA's workload

• "Current different supervisory cultures" in the EU are creating work for EIOPA, noting that "our feedback can sometimes be challenging". Wonder who they're getting at there, France!
• They appear to be developing a Supervisory Handbook of good Solvency II practices. Chapters are already written on Risk Assessment, Boards and Governance, PPP and proportionality in Key Functions to name a few. Something to look forward to no doubt

Ireland-specific

• Go out of their way to applaud the onerous reserving governance in Ireland (here?), considering it practice which other countries should emulate.
• On the other hand, "...in the specific case of the Irish insurance market" he targets the country from a conduct risk/General Good perspective as one of the main cross-border players in the Union.

One can't help but feel the latter part of the speech was deliberately laid at the feet of the Irish, rather than aired in a more generic manner, given the country's continued corporation tax-related appeal to cross-border distributors. I guess when it comes to identifying the perfect audience to sketch out EIOPA's inevitable foray into Conduct Risk regulation, Sr. Bernadino found what he was looking for...