How well are we “doing” with our overall security program?

By: Edward Liebig

07/14/2023

Additional Complexity that needs to be measured

The diminishing “airgap” and proliferation of systems designed on IT principles reach deeper levels of the Purdue Model, Operational Technology (OT) operators find themselves gaining additional benefits coupled with additional security concerns. This has led many OT operators to look at OT security through a dedicated “OT Secure Operation Center (SOC)” that focuses on industrial control systems. The OT SOC amasses Tactics, Techniques and Procedures (TTPs), and OT-specific tools to aid the OT Engineering team with defending and preserving their world. These OT SOC(s) concatenate metrics and Indicators of Compromise/Attack (IoC/IoA) with the IT SOC. Ideally, another layer should amass data on the performance of the security program and its effectiveness. To manage cybersecurity risks effectively and maintain a strong defense posture, organizations need a clear understanding of their security program and the ability to measure their progress toward key objectives.

Enter key performance indicators (KPIs), these metrics allow organizations to gauge and track their cybersecurity effectiveness. In this article we delve into cybersecurity KPIs, exploring their meaning, showcasing real-world examples of cybersecurity metrics, and highlighting the numerous benefits of employing a data-driven approach to cybersecurity management.

KPIs in Cybersecurity

Key Performance Indicators (KPIs) are crucial components that allow businesses to measure and analyze their performance in relation to defined objectives. Effective KPIs drive strategic goals, facilitate decision-making, and ensure that your team is on the right track. Here are some of the most pertinent KPIs within the cybersecurity realm:

• Number of Detected Threats: This measures the volume of threats identified within a given timeframe. It provides insight into the effectiveness of your intrusion detection systems and threat intelligence capabilities.
• Incident Response Time: This metric measures how quickly your security team can identify and respond to a security incident. The faster the response, the less potential damage.
• Patch Management Cadence Trend: This KPI determines the efficiency at which identified vulnerabilities are patched. Rapid response reduces the risk of exploitation.
• Percentage of Staff Trained in Cybersecurity: This indicates the proportion of employees who have undergone cybersecurity training. A well-educated workforce is the first line of defense against cyber threats.
• Cost of a Data Breach: This KPI provides a tangible figure on the financial impact of a “declared” data breach, factoring in costs associated with recovery, regulatory fines, reputational damage, and customer compensation.

Benefits of these KPIs include:

• Improved decision-making: Real-time, data-driven insights allow for more informed, timely decisions.
• Enhanced communication: Clear, measurable goals improve transparency and align the entire organization on cybersecurity objectives.
• Strategic alignment: KPIs help ensure that day-to-day activities are aligned with the larger strategic goals of the organization.

Challenges to measuring and automating the collection and analysis of these KPIs include:

• Data integrity: Ensuring accurate, reliable, and consistent data can be challenging, especially with large volumes of data and numerous data sources.
• Timeliness: Real-time or near real-time data collection and analysis can be difficult, especially for large organizations with numerous potential threats and vulnerabilities.
• Complexity: Cybersecurity is complex, with many interconnected parts. Simplifying this into easy-to-understand KPIs can be challenging.

In cybersecurity specifically, KPIs help to evaluate the effectiveness of security measures and processes to mitigate security threats.

Some other commonly used KPIs in cybersecurity:

• Security incidents. This KPI measures the total number of “declared” security incidents, including data breaches, malware infections, unauthorized access attempts, and system compromises.
• Intrusion attempts. How often have malicious actors tried to breach your networks? By monitoring “who’s knocking at the door” it feeds intel to identifying and profiling potential threat actors.
• Mean time between failures (MTBF). How much time elapses between one system or product failure and the next? This metric helps to understand system reliability.
• Mean time to detect (MTTD). MTTD measures the average time taken to detect security incidents from the moment they occur. It reflects the efficiency of an organization’s detection mechanisms and its ability to identify and respond to threats promptly.
• Mean time to recovery (MTTR). How long does your organization take to recover from a system failure?
• Cost per incident (not Breach). This KPI measures the average cost incurred by the organization for each “declared” security incident. It considers factors such as incident response efforts, investigation, remediation, legal actions, regulatory fines, and other associated costs.
• Cybersecurity awareness training. How well are you maintaining documentation for security awareness training? Are you including all members of your organization, including senior executives?
• Number of cybersecurity events reported. Are employees and users reporting cybersecurity issues to your team? If yes, that’s a good sign; the employees and stakeholders recognize the issues outlined in your training.
• Compliance with security policies and regulations. This compliance metric measures the degree of adherence to security policies, standards, and regulatory requirements. It helps to assure that security controls and measures are implemented as per the defined guidelines.
• Security ratings. Security ratings provide a simple score to communicate metrics to non-technical colleagues, evaluating your company’s security posture in various categories such as network security, patching cadence, endpoint security, IP reputation, web application security, hacker activity, leaked credentials, and social engineering.
• Phishing attack success. This KPI measures the percentage of employees who fall victim to phishing attempts by clicking on malicious links or providing sensitive information.
• Vendor patching cadence. This vendor risk management KPI refers to how often (and promptly) third-party vendors release and deploy patches to address security vulnerabilities in their products or services. It is an essential aspect of third-party risk management as it directly affects the security of organizations relying on these vendors.

By critically looking at your organizational goals and employing these or other KPIs, you can better manage, understand, and enhance your organization's cybersecurity posture. Even amid adding an operational technology, OT SOC to the mix.