search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.48k stars 8.04k forks source link

[Security Solution] Splitting the package with prebuilt rules #187648

Open banderror opened 3 days ago

banderror commented 3 days ago

Epics: https://github.com/elastic/security-team/issues/1974 (internal), https://github.com/elastic/kibana/issues/174168

NOTE: This might be converted to an epic when we have more understanding of the scope and requirements.

Summary

Recently we had an incident in Serverless where Kibana instances would crash with an OOM because of an installation of the security_detection_engine Fleet package that Security Solution uses to distribute prebuilt detection rules. Fleet loads whole packages into memory before installing their assets, and this package had become too big for that. The incident has been mitigated by temporarily decreasing the number of assets in the package by ~50%.

Mid-term measures for the 8.16 release cycle will be stream-based package installation and smart limits for it.

As a long-term measure, we should consider splitting the package with prebuilt rules into a few smaller packages.

elasticmachine commented 3 days ago

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine commented 3 days ago

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

elasticmachine commented 3 days ago

Pinging @elastic/security-solution (Team: SecuritySolution)