[ Security Solution ] Create a Rule Overview page with built-in visualizations to improve Rule Management

[leandrojmp]

Describe the feature:

Create an overview page with metrics and information about the installed rules on the cluster.

The metrics would be things like:

• Number of rules by severity (low, medium, high etc): Could be a metric visualization showing the number of rules for each severity and using the same colors as the background of the visualization (green for low, yellow for medium etc)

• Number of rules by type (query, ml, esql etc): Could be another metric visualization showing how many rules of each typ exists.

• Number of rules with actions (webhook, slack etc): Could be a visualization showing how many rules have actions or not, and how many rules have each type of action related to the connectors. For example, how many rules have a Webhook Connector.

• Distribution of the rules by schedule (5m, 10m, 1h etc): Could be a histogram where the x axis would be the schedule times and the y axis the number of rules in each schedule, or maybe a heatmap.

• Distribution of the rules by Elastic or Custom: A simple pizza visualization showing how many rules are from Elastic and how are custom rules, showing the percentage over the total.

• Distribution of the Rules by the default tags (Domain and Data Source): Could be a visualization showing the coverage of rules by Domain and Data Source, also maybe by integration. Similar to the MITRE covarage

Currently even filtering those things are not possible to do in the rules management page and keep track of the rules is become something more and more complicated.

Describe a specific use case for the feature:

Besides the normal use case of management the rules and keep tracking of what we have in the cluster, it is pretty common to get answers from management and even external audit about our detection rules.

For example, the following questions cannot be answered just by looking somewhere in Kibana:

• How many high and critical (or low, or medium) rules we have?
• How many Machine Learning rules we have? How many ES|QL rules we have?
• What is the schedule for our rules? How many are executed every 5 minutes? How many are executed every X minutes?
• How many rules are sending the alers to the output X?

One workaround is to use specific tags in each rule and filter by tags in the rules management page, but this is far from ideal since the filtering in the rules management page is not good, if you filter by 2 tags, it will be an AND not an OR, and also you need to keep scrolling to find the tags.

Another workaround, which we are using, is to use the Kibana detection rules API to extract the rules, parse it, and index in a specific index so you can create dashboards with the information you need.

I think that having a Rule Overview page is something that should be a core feature of the Security Solution.

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[approksiu]

Thanks for the issue @leandrojmp, this is aligned with our roadmap