[Security Solution] Make index patterns of prebuilt detection rules changeable

[MarcusCaepio]

Hi all, at the new detections feature, the imported detection rules are all configured with the default SIEM index patterns:

Also, when you change the SIEM index patterns in the Settings->advanced section, the detection rules are not updated.

So, in our case, where we save cisco logs in a "cisco-*" pattern, we cannot use the detection rules. It would be nice to let the imported rules assume the changed SIEM settings, too. The only other way would be, to duplicate and change an imported rule, which is not meaningful.

Thanks and cheers, Marcus

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[TRISAF]

Hi Team, any news on that? we are in a POC and really need this feature to think to buy Platinum Xpack. it is in pipeline? Could be usefull make an intermediate astraction or tagging for index patterns and rules to match same tagging and not indexes?

[MarcusCaepio]

This issue got no attention since february so I don't think, they are working on this. I am also struggling with this, while I am building my new elastic cluster ;(

[rbr101]

This is also an issue for me, this should be configurable (MUST). It's not an option to duplicate and change every single rule. Also how to change this for the ML jobs? I think this is not possible.

/R

[MikePaquette]

Thanks @MarcusCaepio and all for the continued feedback regarding SIEM/Security app capabilities for customizing prebuilt detection rules. (i.e. those rules provided by Elastic, documented here).

This is a complex challenge for which a good solution needs to balance the ability for pre-built rules to be updated by Elastic's Security Team, and for the rules to have certain attributes customized by each user for their own environment.

As you've discovered, currently, only rule actions and exceptions are allowed to be customized for prebuilt rules.

A prebuilt rule's selection of index patterns is part of the rule definition, and is not allowed to be edited by the user. This creates a challenge when users would like to apply prebuilt rules (maybe a large number of them) to their own data, which might be indexed in an index pattern that is different than the default set of index patterns included in the rule definition.

The security team understands this challenge and is currently investigating multiple approaches to address it, including a capability to perform bulk edits of detection rules, ability to edit additional fields (that survive updates) in prebuilt detection rules, and simplified rule management that allows better searching, filtering, and operating on of the set of installed rules. We can't yet provide specific details about when such improvements might appear.

We don't think that automatically updating the rule definition's index patterns with the securitySolution:defaultIndex list is a good solution because most rules no longer use that list of index patterns, and it could have unintended consequences for detections. The variability of rule index pattern usage is evident in a recent sample of prebuilt detection rules.

Today, users must either:

1. change their data ingestion architecture to use index patterns already selected by prebuilt rule definitions. (This is hard, and often data ingestion is not controlled by the security analyst/rule_author.)
2. Manually duplicate and edit prebuilt rules to change the selected index patterns used by the rules.

Assuming 2 is the most practical workaround, users must: 1) Decide which prebuilt rules they'd like to modify to use their new index pattern 2) Duplicate those rules 3) Export the duplicated rules using the bulk export function into NDJSON file 4) Edit the NDJSON file to modify the "index" 5) Re-import the rules

Also @rbr101 is correct, this is not currently possible for detection rules that are based on machine learning, since the ML job data feeds have their own index patterns specification, and must be cloned/updated separately.

In the case of the original post, with a cisco-* index pattern, depending on what kind of Cisco logs are included, it might make sense to find all prebuilt rules with the network tag, duplicate them, export the duplicate rules, edit them substituting the index pattern as follows

and then reimporting them

Thanks again for your feedback.

[Foxboron]

We currently have Platinum Xpack and need this feature to simply our deployment setup. Current challenges is that the SIEM detection rules uses outdated product names from several vendors.

Is there any progress on this? Is there any similar features in the pipeline?

[begin-thread]

Hi team and @MikePaquette ! This ticket is very very well documented. Do you know if there is any update?

I have the same issue as I created a dataset in a specific index for students to work in. Then I can export and import back the snapshot from a S3 bucket easily.

But, I need duplicate each rules I want the dataset to be run into.

Thanks!

[banderror]

Hi @begin-thread and thanks for your feedback! We're actively working on adding support for customizing prebuilt detection rules to Kibana. The best place to track the progress would be this ticket: https://github.com/elastic/kibana/issues/174168.

When this feature is live, you will be able to update (customize) prebuilt rules' index patterns by either editing rules individually, or bulk adding/removing index patterns from them (related docs).

Let me know if you have any other questions or feedback!

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[begin-thread]

Thanks @banderror for the response. I really appreciate. Can't wait for the feature it will be so usefull!