3 Challenges to Industry Benchmarking
It was an honor to be invited by the Capital Factory to meet with CISA’s Infrastructure Security team and address how to scale cybersecurity programs. I enjoyed giving a brief presentation on SecurityGate.io’s mission and how we are serving the critical infrastructure community.
One of the questions I received was about industry benchmarking and if we see a demand for it within our market space. It made me realize there are a couple limitations, and I wanted to put them out there and facilitate a discussion around them.
1. How do we measure people and process?
When conducting an assessment, it is generally easy to answer technical control questions. I find it more challenging to get honest and validated feedback on the people and process side, but identifying and remediating gaps in these areas is very powerful towards increasing cyber security. Because of that, I think that sort of benchmarking can be a challenge based on the nebulous nature of many assessment framework questionnaires.
2. Who wants the benchmark?
In some sectors like utilities, a standard such as NERC CIP is generally managed by a governance or compliance group – not so much the security folks. A focus on compliance versus security limits a lot of requests for benchmarking because in a compliance-oriented scoring system, you’re comparing yourself to the hundred percent. Benchmarking on cybersecurity maturity is challenging sector to sector based on the various personas involved in compliance or security.
3. Do resources drive maturity?
Even to someone new to cybersecurity, it’s generally understood that if you have more time, budget, and people to focus on security controls, then your overall maturity should be increasing. Even if you find you are under-resourced, there are ways to gain efficiencies that won’t break the bank and even help your security teams get some time back from doing risk assessments.
So, here’s the question: how do you benchmark upper-middle market to middle-market to SMB market within the same sub-sector? We’ve worked with several OT/ICS security experts and leaders from each of those categories and have seen a common desire to seek efficiencies in risk assessments as well as fixing gaps. They’re not doing it alone – there is a strong sense of common defense out there – so while benchmarking might help them see how peers are doing, they really want to know more about those gaps in their own organization to schedule and tackle in a methodical manner.
We already confirmed in point #1 that each sector may need its own comparative benchmark standards. What about company size, or even better, company resources? Would that mean there are 16 different bench markings and three different sub-benchmarking scores for CISA? That could really get complex.
The Ideal Benchmarking System
Despite all the challenges in front of us, we need a scalable way to benchmark security by sector, and we need it fast. I believe the fastest way to achieve a benchmarking trend that is self-governed by private companies is to encourage, enable, and enforce a common language. We don’t necessarily need more or fewer frameworks, but rather focus on controls languages and the domains they fall within as a minimum standardization across the industry.
