elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.76k stars 8.17k forks source link

[Security Solution][Alerts] Warn if rules are querying indices without efficient range configuration #120687

Open rylnd opened 2 years ago

rylnd commented 2 years ago

Summary

We have seen some customers experience issues when using fields other than @timestamp to sort their rule queries. While https://github.com/elastic/elasticsearch/issues/81457 addresses this issue on the elasticsearch side by exposing index configuration, on the kibana side we'll need to update the detection engine to compare the rule's timestamp field to the relevant index configuration, and warn them if it's missing/incorrect.

Acceptance criteria

Related issues

https://github.com/elastic/elasticsearch/issues/81457

elasticmachine commented 2 years ago

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine commented 2 years ago

Pinging @elastic/security-detections-response (Team:Detections and Resp)

javanna commented 1 year ago

We have seen some customers experience issues when using fields other than @timestamp to sort their rule queries.

Do you have more info on what those issues were? Was it a case of shards being skipped after all, or shards that had relevant documents and needed to be queried? We have discussed the linked ES issue and we decided not to implement the feature for now, but we'd reconsider if we collect more evidence around the need for it.