Last year the Dutch were shocked by the events during the annual celebration of the Queens birthday on April 30th. During the celebration in Apeldoorn which was visited by the Queen and her family, a car ploughed into a crowd killing five people, wounding 12. The 38-year-old driver was targeting the royal family, his attempt failed. In the months after the event it was investigated why this attack wasn’t detected by the extensive risk assessments of the various security agencies involved and why the risk mitigation strategies where not effective enough to either prohibit the attack or at least safeguard the Queen, her family and the people taken part in the celebrations. To my opinion one of the reasons is the method used to do the risk assessments. While reviewing the reports on the risk analysis and the identified risk mitigations of the event I came across an old friend (or is it fo?), the well known risk matrix. The method is used a lot, but this doesn’t imply that it is an effective risk analysis method, let a lone a sound basis for developing risk mitigation strategies. Here is why.A risk matrix is a table that has several categories of probability and impact. Each cell of the matrix is associated with a recommended risk mitigation strategy. The matrix calculates risk as the product of probability and impact. Note that risk is not a measured but a derived attribute (Risk=Probability*Impact). The cells in the matrix are coloured to indicate the severity of the risks. Red for the highest risks, green of blue for the lowest risks. The matrix offers an easy to use and straightforward way to organise pre-listed scenarios in terms of risk. Its use has spread through many areas of applied risk management consulting and practice. It has even become part of national and international standards. Also it used at the Dutch Ministry of the Interior that is responsible for the Queen’s safety and at the National Coordinator for Counterterrorism which is responsible for policy development and coordinating anti-terrorist security measures. Using a risk matrix doesn’t require any training to explain or apply it. It looks nice with its intuitive colouring. Some people really make an effort in developing wonderful colourings to impress, but this cannot hide what’s wrong with risk matrices as recent research by Tony Cox proves.
In short there are four major shortcomings. First of al the risk matrix uses ordinal scales, for example the probability of a scenario is Extreme, High, Medium, Low or Negligible. The consequence of using ordinal scales is that the scenario’s can only be arranged in either increasing or decreasing order of probability or impact. It is impossible to say that a scenario is twice the probability of another scenario, how than can one decide on the risk of the scenario and decide on the right mitigation strategy? Compare this with the ratings of restaurant or movies. Using the star rating you can decide which restaurant to go to, where dinner at a 4 star restaurant is surly better that at a 1 star restaurant. 4 meals in a 1 star restaurant won’t make up for a dinner in a 4 star restaurant however; these ratings cannot be added or multiplied.
Second shortcoming is the low accuracy of the scales. When 4 categories of probability are used each category takes 25% of the total scale, this means that scenario’s with probability of 51% are in the same category as scenario’s with probability 74%. Quite a loss of detail, while risk management is about details! Third and very serious shortcoming is the assumption that scenarios are independent. Consequence is that in case of correlated scenarios the joined risk of the scenarios is ignored. When the scenario “Attack on the Queen’s bus” was combined with the scenario “Car ploughing into a crowd, breaking the barriers” maybe the April 30th attack could have been prohibited. Finally the risk matrix results in an inconsistent ordering of risks. Scenarios with an equal risk profile are placed into different cells of the matrix leading to different risk mitigations. Very serious shortcoming I would say. To explain see the figure. In the risk matrix curves are shown which connect all possible scenarios with equal risk. As you will see, the curves run through different coloured cells which should not happen, leading to inconsistent ordering of risks and therefore different risk mitigation for the same risks. A doctor takes the Hippocratic Oath promising to not hurt his patient while applying treatment. The same should apply for risk analysis methods. Be sure that the method applied is effective and really reduces risk (prove it!). Given the above shortcomings it will be clear that risk matrixes will introduce risk instead of reducing it. Be well aware of that next time when you encounter one. I would suggest switching to another risk analysis method, one that is consistent and fact based.
