Risk Analysis Placebo’s

Carbon Capture and Storage (CCS) has much attention in the Netherlands. It is thought of as one of the ways to meet the climate targets. It’s the Dutch government’s intention to strive for a 100% sustainable energy system. For the Dutch government, CCS offers a solution for the period of transition from fossil fuels to sustainable sources of energy. The government wants to start a CCS pilot project in Barendrecht in which CO2 is stored in an empty gas field. A mandatory risk assessment for the Barendrecht project claims CCS to be safe. Risks of a sudden blow-out of concentrated CO2 would be minimal, with the chance of a mortal accident of less than 1 in a million years. I took a closer look at risk analysis method that was applied and found that is incomplete and unfounded. The calculated risks are underestimated leading to a false belief that it is save to commence with the project. The flaws I found do not only apply to the risk assessment of the Barendrecht project but to many risk assessments of storage or transport of dangerous goods. These risk assessments are therefore Risk Analysis Placebo’s.

The flaws I found are numerous. One of the most important flaws concerns the validation of data used in estimating probabilities like failure rates. In assessing risks it is required to know the failure rates of for example pipelines, compressors, etc. In the Netherlands these parameters are prescribed by law. It is known that these standards are not verified often enough. Also knowledge from accidents is not used to update them. This already is bad practice from a risk management perspective. For the risk assessment of CCS it is even worse. The failure rates for the CO2 pipelines were derived from failure rates of other installations; they were not verified at all. Since CCS is a new technology this is quite questionable. Would you launch a space shuttle for the first time, using the model of a sputnik for test runs?

Another important flaw I found concerns the model used. Besides the standardized parameters, the model to be used for a risk assessment is also prescribed by law in the Netherlands. I’m still wondering why a one size fits all approach would be sufficient in risk assessments. The prescribed model simulates the cloud of the dangerous gas or substance after release under different circumstances. Given this estimated cloud the concentration of the substance can be calculated at different distances from the source. Risk is then calculated at each location by multiplying the estimated number of casualties with the probability of the event. The prescribed model has several flaws. First of all it is not capable of dealing with buildings. It assumes that the area for which the risks are assessed is completely flat. Also it is not capable to deal with wind speeds lower that 1,5m/s. In case of CO2 no wind is the most dangerous scenario as the Monchengladbach incident in 2008 has shown. To my opinion the most important shortcoming is that for many parameters in the model an average estimate is used (see also What’s wrong with average). To name a few, an average is used for failure rates, population density, pressure in the pipeline, amount of substance released, wind speed, temperature and diameter of the leakage. As Sam Savage explains very well, using an average can be a great mistake. Since the model for the mandatory risk assessment for sure is not linear the strong form of the Flaw of Averages (=Jensen’s Inequality) applies. In such a case no average input (=point estimate) must be used but a simulation must be run using the complete probability distribution of each of the parameters to get the true average output of the model and therefore the correct risk measures.

The above flaws in the risk assessment method require change as soon as possible. More sound risk assessment methods are needed, the current one feeds risk analysis placebo’s to the public and the decision makers. The current by law standardized risk assessments in the Netherlands have little value for estimating risks, let alone developing sound risk mitigation strategies. Risk analysis goes beyond plugging a few numbers into a model. It requires thorough knowledge of the situation being assessed, tested and validated models and validated data. Work of experts, not only in engineering in risk analyses as well!

For Dutch readers: A news item for television was made out of my analysis, see Netwerk.