If you’re in the cybersecurity business, you’ve likely invested a fair amount of time (or perhaps a lot of time) in generating cybersecurity metrics. And you may have experienced some of the frustrations that I covered in my recent article, Cybersecurity Metrics: Avoiding Common Pitfalls. In this follow-up article, I’ll delve into what it takes to create meaningful cybersecurity metrics that communicate effectively with your stakeholders and provide useful measurements of your security program’s impact. 

1. Know What You Want to Accomplish 

Creating meaningful cybersecurity metrics starts with defining the objectives, goals, and activities relevant to managing technology risk and supporting business objectives. A security program aligned with an industry-standard framework to establish a comprehensive set of objectives will help define the metrics that you want to establish. The National Institute of Standards and Technology Cybersecurity Framework (NIST CSF) Categories and Subcategories are written in language that can easily be used as objectives for the security program. 

2. Talk to Your Stakeholders  

The next step in creating metrics is to consider the audience intended for each metric. As a security practice leader, your stakeholder audience may include business peers, organization leadership, internal and external partners, and technical contributors. Each audience will likely require varying levels of complexity and technical detail. Meet with individual members of each of your stakeholder groups to review security program objectives and proposed metrics so they understand how the metrics help evaluate progress. Validating assumptions around proposed metrics not only helps ensure they’re relevant for each stakeholder audience but also helps garner program buy-in. Leadership stakeholders may not find all of the metrics used to manage security program operations meaningful. However, sharing how these metrics are tracked will help establish credibility that the underpinnings are well run. 

That said, metrics that leadership stakeholders will find useful are typically associated with implementation progress, investments in improvements, and program operation costs. Cybersecurity is a component of continuous risk management; therefore, progress related to developing and implementing the security program is very important.  

3. Measure Security Practice Maturity 

Goals associated with implementation timelines provide opportunities to measure progress and the evolution of maturity. Capability maturity models provide useful ways of keeping score for the implementation of security program functions. These types of metrics are applicable to leadership audiences that need to know how the security program is performing without getting into the weeds of overly technical aspects. The NIST CSF provides the functions, categories, and detailed subcategories with built-in levels that provide a solid basis for the program. 

When it comes to evaluating maturity, it is not wrong to forego quantifying objectives measurement. However, it’s important to clarify the desired goals, objectives, and associated capability behaviors and patterns at each maturity level, typically better defined with words rather than numeric levels.  

At the same time, it is fairly common to utilize a capability maturity model that has numeric levels. Metrics that are quantified numerically are better suited than the Goldilocks standard of high, moderate, and low values or stoplight red, yellow, or green levels, especially if these levels are not well defined. It isn’t wrong to use them because they may be better suited for the audience than too many raw numbers. Just ensure they are meaningful to the audience with clear definitions of what constitutes each level. Quasi-quantifying by putting levels into the form of numbers can be useful, but these qualitative descriptions shouldn’t get confused with quantified measures.  

4. Security Tools Are Not a Silver Bullet 

Tools can provide a lot of data, but without defining the security program objectives and tying the tool data to a meaningful metric, how will you know if the tool is helping to reach the objective? Asking questions about the security program and defining measurements to answer those questions is the basis of defining metrics. These questions and answers can be constructed for each of the objectives of the program. What is the trajectory? Are there any trends? Are you getting better, holding ground, or getting worse? 

5. Consider Measuring Security Program Operations 

When you operate the security program using a services model, then metrics used for running a business can be useful. Time is money, and resource utilization is not a sunk cost, as viewed internally within many organizations. Time and effort to operate the security program functions should be tracked, and metrics associated with either improving efficiency or understanding capacity can help ensure that the organization makes better decisions. Isn’t that what metrics are supposed to help do? Cost of operations is a good metric that requires tracking effort, prioritizing activities, and measuring the gap for items that are deferred or delayed. 

6. Tracking Time-Based Metrics

Metrics associated with time can be useful. For example, when detecting potential attacks and malicious activities, it’s worth capturing the amount of time from occurrence to detection. Understanding where improvements are needed is a vital aspect of ensuring that detection and response times fall within the organization’s risk tolerances. Finding and addressing vulnerabilities within appropriate timeframes helps reduce exposures. 

7. Do Your Own Research 

Plenty of reference materials on metrics are available, and I encourage you to use them. You will find some shortcuts, helpful online forums, and even some worthy rabbit holes to explore a variety of specifics and interesting parallel topics. Just apply what you learn within the context of your own program goals. Over the course of my career, I have leveraged materials specific to cybersecurity as well as broader business practices. 

Go Forth and Measure 

If a metric helps you achieve your goals and objectives, it is probably a good metric. During the initial process of defining security program metrics, there are many detailed metrics that may only be viewed by the team managing the program. Tracking them internally allows you to test them and see how the story evolves. It can be a little messy but give your metrics a chance to evolve before casting them aside or presenting them to stakeholder audiences.

Remember, metrics should be used to help clearly communicate. Some metrics may answer questions for other metrics you are trying to gather. The metrics you need to manage the security program may only be useful for you and not be the metrics that you share with other stakeholder audiences. Make sure your metrics are communicating clearly to your audience, answering questions about the operations and effectiveness of the security program, and providing more clarity than obscurity. 

A final note: if you need help with creating meaningful security metrics, get in touch or check out our Security Breach Protection program and complimentary Cybersecurity Workshop to see if this might be the right fit for your organization.