When an HTRA Gives Poor Results
There are a number of threat and risk assessment methodologies available to information system security practitioners. The Harmonized Threat and Risk Assessment (HTRA) is one such methodology that has been used in the Federal Government of Canada. Like many other similar methodologies, it identifies threats, enumerates assets and the impact of their loss or compromise, and considers vulnerabilities. Risk is computed through a number of translation tables using the compiled threats, injuries, and vulnerabilities.
If your information system already exists and you want to know what business risk you are taking by relying on it, then the HTRA can be a very useful tool. This is because the HTRA is "asset-centric". A large portion of the assessment activity is often devoted to asset discovery and there are representative tables of assets to help you get started. The results of an HTRA on an existing information system are neither good nor bad (although they may not be welcome). They are just a point-in-time observation on the current risk posture of the system.
Unfortunately, because it is seen as an "asset-centric" methodology, the HTRA is often applied incorrectly when dealing with new information systems (or significant changes to existing systems). The problem is that the HTRA is only called upon when there are assets to assess which occurs late in the project. This will often (but not always) lead to poor results for the project. In some cases, good results are achieved because project staff involved in design are aware of existing threats and appropriate safeguards and are forceful in asserting security. More commonly, the project does not have someone conversant in IT security who can insist on safeguards. Using the HTRA on a new information system by waiting until it is substantially complete before executing can rob the project of many significant risk management opportunities during the project lifecycle.
An Abstract Example
Imagine a city municipality tasked with placing a new traffic bridge over a waterway. Now try to imagine doing this without an engineering firm involved in the project. As a final stretch, imagine that project has successfully managed to erect a bridge using some architecture patterns and best practices.
Now imagine the municipality hiring an engineering firm to assess its work. Among other things, the firm considers all of the local natural hazards and serious adversarial threats. In doing so, it determines that earthquakes having magnitude 5 on a Richter scale are relatively frequent in the area. It analyzes the bridge with this information in mind and determines that it will collapse the next time such an earthquake occurs. Because the municipality considers loss of life to be a very high injury, the overall risk is high. In the report, the firm recommends increasing the centre column thickness or demolishing the majority of the bridge and replacing it with a two-column variant. The project determines the cost of these recommendations to be 60% and 140% of the original contract value, respectively.
What are the municipality's "risk management" options at this point?
This is a ludicrous example but its information security analogue seems to play out every day.
In reality, an engineering firm would be contracted to erect the bridge at the outset. One of their first steps it would take is to conduct an environmental threat assessment in order to determine which threat events are applicable and worth considering in the context of the bridge. Why? Because these are the threat actors and events that the bridge must be built to withstand. The City could decide to accept high risk from magnitude 5 earthquakes and save lots of money on construction costs. But in North America, that will land you in jail because building codes don't allow substandard structures to be erected.
Notice that no "assets" exist yet but a TRA has already been done. A number of adversarial and non-adversarial threat actors have been identified in the context of the bridge and the potential for each to cause injury has been assessed. For example, a magnitude 5 earthquake will stand a good chance of toppling a bridge that is not designed to withstand it and that will result in loss of life. Risk is high so the engineering target is to build a bridge proof against magnitude 5 earthquakes. In an alternative scenario, a deliberate threat actor could detonate a powerful bomb on the bridge or against its supports which might reasonably be expected to cause loss-of-life. However, the bridge is unremarkable in all respects so this is not a likely occurrence and the risk is low. No special engineering considerations need be entertained.
The "vulnerabilities" being considered during this high-level TRA are the bridge architectures and potential design patterns. For example, perhaps the initially chosen design involves a typical concrete span with a center column. It may have been chosen for aesthetic purposes of functionality (e.g. easier for shipping to pass below). But the security constraint on that design (to survive a bomb blast at the base of the column) requires the radius of the column to be doubled which will increase costs significantly. An alternative approach would be to install two thinner columns each of which could bear the entire load for a period of time. It looks less pleasing and puts a constraint on the width of shipping traffic passing below but it will fit within budget. Or, a suspension bridge could be implemented and it will survive an earthquake without constraining shipping but it will cost more and will take longer to construct. Etc.
The architectures and designs have become the asset for the purposes of threat and risk assessment. If the project chooses the cheapest, single column span, then we know it will not suffice unless additional changes are made. We can record that fact in an architecture-level TRA. The municipality must become involved to authorize a different architecture with potentially different costs and schedules but acceptable levels of risk. Or, if the money and time are not available, the city can elect to cancel the project before it even gets under way.
This bridge example may seem like a laborious exercise in the obvious. But civil engineering examples tend to be clearer in demonstrating that risk assessment is a part of a continuous engineering process and that that architectures and designs are significant "assets" to be considered within a TRA. Waiting for the bridge to be built before considering the threat environment will rarely result in a trustworthy outcome.
HTRA in the SDLC
The HTRA explicitly calls for the TRA to be exercised throughout the project lifecycle (Annex A-1, Section 3) and, most importantly, have early involvement:
Firstly, and most importantly, TRA processes should be initiated in the earliest phases of a project, at the conceptual or planning stage. Of course, the first assessment will be a high level review because many assets and their associated vulnerabilities cannot be identified until detailed designs emerge later in the project life cycle. Nevertheless, the initial TRA report can influence the direction of a project, helping to identify and avoid riskier options or alternatives.
The HTRA includes the notion of "initial and interim TRA reports" that "identify potentially unacceptable residual risks and suggest cost-effective security solutions before any irrevocable design choices are made by the project team." Again, such reports are not necessarily monstrous documents generated by a parallel security assessment function and lobbed into the project like grenades. Rather, they should be more like the recorded minutes of conversations between architects and engineers structured to demonstrate a sound analysis and presented as background material for a decision brief by the authorizer.
To be effective in the earliest phases of a project, the HTRA analyst has to be comfortable with architecture and engineering concepts in order to be able to critique the evolving design. In reality, there is no need for a separate "TRA Analyst" to be called upon during a project and no need for a "comprehensive" TRA to be performed near the end of the project. Rather, when used properly, the HTRA can be used as one more tool to demonstrate the security view of design and engineering choices.