How well do risk assessments inform decision makers?
Chris Peace, Risk Management ltd, New Zealand
Every day, a newspaper, news broadcast or news website somewhere in the world carries a story about a disaster that might have been avoided by better decision-making.
Were such decisions informed by risk assessments? If so, how effective were those risk assessments in informing the decision makers about the risks?
More generally, what risk assessment process, supported by relevant techniques, should any risk assessor follow to give assurance to decision makers (eg, management, boards, regulators) that they have considered the many causes and consequences of a risk event?
Research to date
A review of academic literature reveals little research that covers how well risk assessments inform decision makers and — specifically — what criteria decision makers use to determine whether they can trust a risk assessment. Some research touches on these questions, but it relates particularly to project management and software development.
We have ISO 31000:2009, ISO 22301, OHSAS 18001 — and the COSO enterprise risk management document. We also have risk analysis documents, setting out guidance on how to carry out a risk assessment. But there seems to be little research investigating how and how well these work in practice. Do decision makers get the information they need?
At this point you may say: “But I use techniques A, B and C”. The point is there has been little research to find out which techniques are used by which groups of risk practitioners in the broadest sense. Yes, we have an international standard IEC/ISO 31010:2010 Risk management: risk assessment techniques that sets out 29 methods, but research has found at least double that number that are not in the standard.
So questions arise, such as: do all risk assessments cover the context or business environment? Do such risk assessments include risk criteria (or, for some safety practitioners, ALARP)? Which techniques are used to decide who to communicate and consult with?
Which risk assessment techniques are actually used? About half the practitioners I meet use a mixture of professional judgement and some form of 5×5 consequence likelihood matrix. But I live in New Zealand and work in the Asia/Pacific region. There are lots of people I don’t meet, so limiting my first-hand knowledge. So which techniques are being used worldwide in risk assessments?
By one count there are more than 40 international standards that relate to risk assessments and thus to decision-making. Who uses which ones?
Does it matter if we don’t know who uses which techniques? Perhaps not, but if we don’t know, how can standards-writers know which to include or exclude? How can educators and trainers know which to teach? Perhaps most importantly, how can risk practitioners use the most appropriate techniques to gather the best available information for decision makers?
The results so far
An online survey of risk and safety practitioners has now been running for a few weeks and some patterns are starting to emerge.
Decisions are often supported by risk assessments, but uncertainty and the context are not always mentioned or discussed in detail. However, stakeholders are usually consulted as part of risk assessments.
A simple majority of respondents have said they use named risk techniques, suggesting that many do not. Risk criteria are not always developed or used.
The more respondents to the survey from other countries and professions that respond the more will be known.
Take part in the survey
If you would like to help answer such questions please would you respond to an anonymous, online research survey being run at Victoria University in Wellington in New Zealand?
The survey is open to anyone whose work involves any part of decision-making, management, safety management or risk management and you should find it professionally interesting and stimulating. At the end of the survey you can download a copy of the questions and your answers and use them to help with CPD (a bonus for IOSH members).
No information will be identifiable to an individual. If you’re ready to start please follow this link.
PS: if you like the survey, copy the link to a colleague.