How to Measure the Effectiveness of Your Cybersecurity Program

February 9, 2017

Scoring a baseball game is a tradition that goes back to the sport’s early days. For fans, it is a great way to get more involved, stay engaged during the game, and understand each game’s story. In the end, every ballgame provides meaningful statistics that make it easy to know how well a team and its players performed during the season. Unlike baseball, Cybersecurity performance is not easily measured. In fact, most organizations would be hard-pressed to pull together a consistent set of statistics that tell the whole story. Why? The challenge is that Cybersecurity metrics lose meaning outside of their specific context and don’t lend themselves to an ‘apples to apples’ comparison with other environments or organizations.

In practice, these challenges contribute to overestimating or underestimating the effectiveness of Cybersecurity programs. This does not mean there are not ways to effectively measure cybersecurity, both at a tactical and strategic level. However, tactical cybersecurity metrics are challenging as the denominator of the equation is almost always unknown (that is, we don’t know how many actual attempts have been made to get into our network). Further, the statistics generated at the tactical level are relevant to our organization and relative to our previous observations.

At an absolute level, strategic security metrics are about understanding how each piece of a Cybersecurity program fits together, what cybersecurity measures are in place, and the ability to implement controls in a timely fashion.

Strategic measurements begin with a holistic view of Cybersecurity across three dimensions:

  • People – understanding how the people in the organization work
  • Process – ability of processes to deliver consistent results against desired goals and objectives
  • Technology - the use of reliable technologies to manage security risks

Understanding the Role of People in Security

Security is a holistic property of an organization’s people, processes, and technology that is only as strong the weakest link in the chain. The weakest link determines the entire system’s level of security. It should come as no surprise that people are often the weakest link in the chain. Oftentimes people are too optimistic and don’t understand the risks, they assume that the technological safeguards in place will protect them, or they circumvent security measures in the name of convenience. So how can we protect our organizations from these challenges? 

1. Communication
The first step is talking with the entire workforce on a consistent basis and getting their feedback, as well as socializing the rules and controls that are in place. The security team should consider and implement changes to their program based on user feedback. The aim is to drive end user adoption, enhance compliance, and improve cybersecurity posture.

2. Training
The problem with communication is that it does not account for the ingenuity of man when solving problems of convenience. It’s one thing to make people aware of policies and procedures, it’s another to have them adopt your security practices. In one case, a hospital implemented a rolling computer desk for doctors and nurses to use while going from room to room. The computer had an authentication mechanism that looked for the user to be in close proximity of the computer to keep the screen from automatically locking. This was an inconvenience when examining a patient and trying to log their condition. So the doctors and nurses came up with a simple solution – they placed a paper cup upside down on the proximity sensor so that it never locked them out, improving the user experience and defeating any security benefit. Technology cannot fix a people problem!

3. Controls
Evaluate security controls through the eyes of the everyday user. When security measures are too inconvenient, people will find ways to work around them. Getting buy-in from your end users is imperative to ensure compliance. If users understand the purpose of the controls and have a say in how they are implemented, they are more likely to abide by the rules and also enforce the controls among themselves.

Measuring Process Effectiveness

Processes are executed through a combination of people and tools. These processes are governed by policies, procedures, and risk priorities that drive security goals and objectives. People and process work together and outcomes should be repeatable with the ability to identify performance trends over time. In a mature organization, each security process needs to be fully developed and integrated with business processes that drive operations.

Defining security roles is one most important aspects of reviewing and refining your processes. Someone who has the complete trust and authority from the executive leadership must be designated as the decision-maker in a time of cybersecurity crisis. Crises require a strict chain of command in order to respond efficiently and effectively. The best way to ensure this is in place is to conduct tabletop exercises with the executive leadership and senior management.

Here are a few critical success factors in measuring process effectiveness:

  • Adopt a maturity model framework to evaluate cybersecurity processes and assign a maturity rating to each group of security processes based on specific criteria (organization vs. industry)
  • Secure leadership commitment to provide resources to gather process data,  use the collected data for decision making, and mature the program over time
  • Invest in data quality to support measurement accuracy and confidence in the measured results
  • Commit to continuous improvement of security processes and find an appropriate balance between usability, security, and cost-effectiveness

Tools and process should work together and be repeatable with the ability to identify performance trends over time. Ultimately, processes should drive the selection and implementation of appropriate tools and technologies.

Evaluating Technology Effectiveness

Selecting and implementing security technology requires two resources that are hard to come by – money and qualified people. Each of these resources is scarce inside of businesses, as well as the cybersecurity community at-large. Therefore, the most important part of measuring technology is proper prioritization of resources.

The critical success factors for measuring Technology Effectiveness are:

  • Defining technology security standards for the organization
  • Agreeing to the best KPIs/Technical measures of risk for the business
  • Measuring effectiveness against defined KPIs

For cybersecurity success from a technology perspective, a balance must be met and an understanding of your environment is required. Not every business requires every technology and not every technology is right for your business. The key to success is doing proper research, understanding how each technology piece works together, and building the infrastructure to support the technology.

While there is no scorecard to measure the effectiveness of a cybersecurity program, it's important to understand the proper application of the metrics you are analyzing. Each type of measurement has its benefits and its drawbacks. Look at how the processes are implemented and used to properly assess their effectiveness. Most importantly, don’t get sucked into a numbers game, as your cybersecurity posture will come up lacking. 

If you enjoyed this post, subscribe to our blog and follow @Idenhaus on Twitter.

Photo credit: Flickr

By going to work quickly to solve the most challenging cybersecurity and identity management problems, Idenhaus takes the pain out of securing corporate information and assets for companies that aspire to maximize their potential in this digital age. Contact us today!

More News