New Cyber Defense Brand DeepSeas to Unite Newly Acquired Commercial Managed Threat Services Business from Booz Allen Hamilton with Security On-Demand. Learn More

Five Questions to Ask When Evaluating a New MSSP or MDR Provider

Identifying and choosing an MSSP (Managed Security Service Provider) or MDR (Managed Detection & Response Provider) can be time consuming and difficult. MSSPs typically have different technology partners, capabilities, and pricing models, which can all have an effect on your overall budget, staffing, and security effectiveness. 

What criteria should you use to narrow down your selection and find the one that is right for your organization?

Here are five key questions you should ask.

5 Questions to Ask When Evaluating a MSSP:

  1. How do you collect, store, process, and analyze the huge amounts of data you bring in?
  2. What security technologies do you integrate with?
  3. If I have an incident, what kind of support will I get from you?
  4. Will you show me the use cases you use for detection and alerting?
  5. Do you have a customer portal where I can view my own data and see the alerts?

1.  How do you collect, store, process, and analyze the huge amounts of data you bring in?

Many providers are going to have hundreds to thousands of customers. That means they are going to be handling billions upon billions of data logs per day.  A good provider will have defined collection processes and provide both log management as well as security monitoring and detection. It is critical that they are able to process, analyze, and query the data quickly, if you are expecting them to find threats in a timely manner.

Most SIEMs a tool for analyzing logs and identifying security threats – are able to generate rules-based alerts, but struggle returning results quickly for even basic queries. They also fail to recognize dynamic, advanced threats, which make up 60% of today’s landscape. In some cases, a SIEM can take days to get an answer to a query, so ask and watch them query real data in the tool, not demo data that is primed to return a result quickly. Also, ask what they are doing to detect advanced, unknown threats.

Another layer that is critical in your solution is a 24×7 Security Operations Center or SOC, which keeps a close eye on your network, and can inform you of any vulnerabilities that are actively being exploited. Ensure that the provider has SOC Analysts round-the-clock who can investigate on your behalf and reduce the amount of false positives the solution generates. Ask how many alerts their customers receive, so your team avoids time wasted chasing false positives. Pro tip: Look for a solution that sends you less than 5 alerts a week.

Adding an effective Managed Detection and Response(MDR) solution can give your IT department, augmented 24×7 monitoring staff, advanced analytics, and automation. Not all MSSP and MDR solutions include behavioral analytics, advanced log analysis, and anomaly detection, but having access to advanced threat detection capabilities will help you find advanced threats in your environment, like ransomware.

2.  What security technologies do you integrate with?

It is not critical that the MSSP integrate with all of your security technologies, in fact, it is likely that most will not completely align. But it is important that they integrate with enough of your appliances to provide adequate security from day one and that the MSSP is willing to work with you to integrate your other devices.

At a minimum, the MSSP must be compatible with your firewalls, IDS/IPS, and anti-virus (not to mention standard syslogs). Ideally, the MSSP would already have an integration with your vulnerability management system, end-point protection, data-loss prevention, and other devices. However, no one service will integrate with all your products, so be judicious in terms of what is a deal breaker and what is not. Just because they don’t today integrate with your end-point protection product of choice, does not mean they are not the right fit.

3. If I have an incident, what kind of support will I get from you?

Few MSSPs moonlight as incident responders, so you should not expect the one you select to be your Incident Response (IR) team. However, you should expect that your MSSP will stand by your side and give you support as you need it. They should be there to pull historical logs, review past alerts, and provide needed information to your IR team.

If your IR launches late on a Friday night, you should expect them to have SOC Analysts available to provide support that night and through the weekend. There are no days off in security operations. Threat actors don’t take weekends or nights off from attacking, and your MSSP shouldn’t take off time from monitoring.

4. Will you show me the use cases you use for detection and alerting?

Effective security monitoring, detection, and alerting should have a heavy dose of reliance on use cases. Too often, MSSPs primarily detect via reputation lists or standard signatures, but do not move much beyond that. Effective MSSPs and MDR solutions employ use cases that can be built as simple detection signatures or even as advanced behavioral analytics. Example use cases include malware beaconing, data exfiltration, privilege escalation, and many more. Essentially, the better MSSPs have built detection rules focused on behaviors and patterns, which provides a higher level of confidence when the alert fires.

5. Do you have a customer portal where I can view my own data and see the alerts?

MSSPs and MDR solutions should strive to be as transparent as possible, after all, it is your data. You should be able to view your logs and dive into the alerts being generated. Even more ideal, though not critical, the portal would include dashboards that summarize your data and the threats being discovered. 

100% visibility into your IT environment is valuable. A portal with dashboards and reporting may help with compliance or internal deduction-making as well. A portal/platform would also give you and your security team quick access to your data, which in the event of an incident response, allows you to employ your own internal threat hunting or external incident response.


There are many other questions and criteria that you will want to consider when evaluating an MSSP and these are just a few of the critical ones. We recommend finding a service that is right for you and provides the greatest level of security. The purpose of an MSSP or MDR is to alleviate the direct burden on you and your organization by providing expert service for something that is very costly and difficult to do in house.

About Security On-Demand

Security On-Demand (SOD) provides full-spectrum threat management and advanced cyber threat detection services for hundreds of businesses and government agencies globally. SOD’s patented, behavioral-analytics ThreatWatch technology enables the detection of advanced threats to protect brand value and reduce the risk and mitigate the impact of a data breach. SOD is headquartered in San Diego, CA with international R&D offices and a Security Operation Center in Warsaw, Poland.