search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.69k stars 8.24k forks source link

[Security Solution] Splitting the package with prebuilt rules #187648

Open banderror opened 5 months ago

banderror commented 5 months ago

Epics: https://github.com/elastic/security-team/issues/1974 (internal), https://github.com/elastic/kibana/issues/174168

NOTE: This might be converted to an epic when we have more understanding of the scope and requirements.

Summary

Recently we had an incident in Serverless where Kibana instances would crash with an OOM because of an installation of the security_detection_engine Fleet package that Security Solution uses to distribute prebuilt detection rules. Fleet loads whole packages into memory before installing their assets, and this package had become too big for that. The incident has been mitigated by temporarily decreasing the number of assets in the package by ~50%.

Mid-term measures for the 8.16 release cycle will be stream-based package installation and smart limits for it.

As a long-term measure, we should consider splitting the package with prebuilt rules into a few smaller packages.

elasticmachine commented 5 months ago

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine commented 5 months ago

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

elasticmachine commented 5 months ago

Pinging @elastic/security-solution (Team: SecuritySolution)