We've told generations of information security students that "security" is all about managing risk but we continue to struggle with exactly how to do that in a field that lacks a long history of actuarial data on just how frequently which can be expected to happen and just how bad it might be when it does. We commonly use qualitative methods based on high, medium, low scales, compliance with best practices, etc, to attempt to demonstrate that we're doing the right things with the scarce resources at our disposal.

Douglas Hubbard takes the rather heretical view that we're doing it wrong and substituting "consensus of ignorance" for quantitative assessment. Hubbard presents his argument in three parts: "Introduction to the Crisis", "Why it's broken" and "How to fix it".

In the crisis introduction he asks some really unpleasant questions about just how much we know about how well the risk management methods we use actually work. Too often we accept the occurrence that "nothing bad happened" as being equivalent to "successfully managing risk" and never ask the impolitic question as to whether we did the right things or were we just lucky that time around (to see how "lucky" you can be, spend some time playing with the Binomial distribution in your favorite spreadsheet program to see how many trials it takes for a risk to be realized at a given probability - the results will likely surprise you). He reviews the current stable of risk management methodologies (beware - many "naked emperors" will be revealed) and sets the stage for the following sections by describing how you might know if a methodology actually worked (assuming you actually looked).

"Why it's broken" delves into how we came to be in such a fix despite the efforts of many bright and conscientious people. With honesty and a dash of humor here and there, the history of the risk management discipline is reviewed and the various approaches dissected. Everyone from management consultants to subject-matter-experts is placed under Hubbard's microscope and found wanting.

He meets the common complaint "but we have no data" head on with a rather snide "you have more data than you think". He notes that we often equate "having data" with "having all the exact numbers" and they are not necessarily the same. In his first book, "How to Measure Anything: Finding the Value of Intangibles in Business", Hubbard makes the point that you always know something about the situation even if it's just a general idea of what the world would look like if whatever was true. Once you have this initial stake in the sand, you can begin to think about taking measurements - not to necessarily give you the exact value but to reduce the uncertainty in your knowledge. This incremental, probabilistic approach to the gathering the necessary data underlies the approach presented in the final section.

By this point in the book, you will have probably seen your favorite approach to risk management lambasted and marked by Hubbard for the scrap heap. In the "How to Fix It" section he identifies three key factors to improve the practice of risk management (p. 202):

• Adapt the language and philosophy of modeling uncertain systems
• Be a scientist
• Build the community as well as the organization

At the conclusion of part three, Hubbard has built an excellent case that we need to bring the quantitative methods back into risk management. We do have issues with the quantity of data but there is more data out there than we think and if we take the empirical, quantitative view of data as the input to a model whose results will be calibrated against reality, we will gain insight into where measurements can be taken to reduce the uncertainty in the data we have.

Though "The Failure of Risk Management" can be read alone, "How to Measure Anything" would be an excellent preface. Though not primarily focused on the information security profession, this book holds much solid advice on how we can elevate our approach to risk assessment and management beyond the high-medium-low rating scales to an empirically-based, defensible basis for decision making. Definitely a recommended read.

Before beginning life as an itinerant university instructor and security consultant, Richard Austin was the storage network security architect for a Fortune 25 company. He welcomes your thoughts and comments at raustin2 at spsu dot edu.