In my profession as an auditor, I get to see a lot of risk assessments. It’s my job to ask questions and to try to understand the results and actions derived from them. The more assessments I see, the more I am convinced that there is something fundamentally wrong with them.
Here is what I observed.
The way organisations establish their risk management practices is heavily influenced by standards like ISO/IEC 27001. This approach consists of two major steps, the risk assessment and the risk treatment.
Risk assessments generally follow a structured process involving three key steps: risk identification, risk analysis, and risk evaluation.
Risk Identification is the first step, where the organization identifies potential risks that have the potential to negatively affect its operations. This involves understanding what assets need protection, what threats they face, and the vulnerabilities that could be exploited.
Risk Analysis comes into play, where the organization examines these identified risks to understand their nature and impact.
This is where I think the flaws begin.
As part of the risk analysis, ISO/IEC 27001 explicitly requires the following:
assess the potential consequences that would result if the risks identified were to materialize;
assess the realistic likelihood of the occurrence of the risks identified
The general idea is to derive a risk level by determining both the potential consequences and the likelihood of an identified risk.
Risk Evaluation is the final step, where the analyzed risks are compared against the organization’s risk criteria to determine which risks require treatment and what level of attention each should receive. This is where decisions are made about whether to accept, transfer, mitigate, or avoid the risk.
The Problem with “Likelihood”
ISO/IEC 27001, is the most widely accepted standard for information security management. Its requirements have a tremendous influence on how organisation of all sizes design and establish their security programmes. As outlined above, the standard explicitly mandates that organizations assess both the impact of potential risks and their likelihood.
This is where the approach fundamentally breaks down. The concept of “likelihood” is inherently flawed, especially when applied to cyber risks and other unpredictable events.
Assigning a likelihood to an event like a cyberattack is inherently challenging, and this difficulty often leads organizations to produce results that are, at best, questionable and, at worst, misleading. The complexity and unpredictability of threats make it nearly impossible to accurately determine the probability of an attack or any other adverse scenario.
Organizations often attempt to quantify the likelihood of risks using qualitative scales, such as “high,” “medium,” or “low.” However, these classifications are typically based on subjective judgment rather than objective data. For instance, an organization might rate the likelihood of an attack as “low” because they have not experienced a significant breach in the past. Yet, this assessment ignores the dynamic nature of the threat landscape, where the absence or occurence of an attack does not indicate a lower probability of future incidents.
Similarly, consider the analogy of winning the lottery. If you play the lottery today and lose, your chances of winning tomorrow are exactly the same as they were today. The outcomes are independent of each other. This principle holds true for many types of risks; the past does not influence the future in a deterministic way.
By focusing on likelihood, organizations may overlook the real and pressing dangers they face.
Instead, they should focus on understanding the potential impact of risks and developing robust strategies to mitigate or manage those risks, regardless of how “likely” they seem based on past experience. In other words, the emphasis should shift from trying to predict when or if a risk will materialize to preparing for the possibility that it could happen at any time.
If you want to learn more about risk assessments in the context of ISO/IEC 27001 you might want to have a look at the following video on my YouTube channel.
