[Fleet] Stream-based programmatic API for installing packages

[banderror]

Epics: https://github.com/elastic/security-team/issues/1974 (internal), https://github.com/elastic/kibana/issues/174168

Summary

Recently we had an incident in Serverless where Kibana instances would crash with an OOM because of an installation of the security_detection_engine Fleet package that Security Solution uses to distribute prebuilt detection rules. Fleet loads whole packages into memory before installing their assets, and this package had become too big for that. The incident has been mitigated by temporarily decreasing the number of assets in the package by ~50%. However, this is a short-term measure that we cannot keep for a long time. We need a fundamental solution to this problem in Fleet itself.

Our idea is to introduce a stream-based API for installing Fleet packages:

• This would be a programmatic API (a method of the PackageClient) available for Security Solution on the server side, and not available to Kibana users via HTTP. Security Solution would wrap this API with its own HTTP API endpoint for installation of the security_detection_engine package.
• This API would use Nodejs streams to avoid loading whole Fleet packages into memory.

We hope this solution would help us prevent spikes in memory usage when installing the security_detection_engine package.

Details

This is where/how Security Solution installs the package on the server side:

https://github.com/elastic/kibana/blob/1040bae64087e2d8fb6a4ef77b93b731b74b8d27/x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/api/install_prebuilt_rules_and_timelines/install_prebuilt_rules_package.ts#L38-L41

The corresponding method of the PackageClient is:

https://github.com/elastic/kibana/blob/1040bae64087e2d8fb6a4ef77b93b731b74b8d27/x-pack/plugins/fleet/server/services/epm/package_service.ts#L71-L76

We would need a stream-based alternative of the ensureInstalledPackage method.

It could be done via adding an option to the existing method:

  ensureInstalledPackage(options: {
    pkgName: string;
    pkgVersion?: string;
    spaceId?: string;
    force?: boolean;
    stream?: boolean; // <-- NEW OPTION, by default is false
  }): Promise<Installation>;

Or via adding a new method:

  ensureInstalledPackageInStreamMode(options: {
    pkgName: string;
    pkgVersion?: string;
    spaceId?: string;
    force?: boolean;
  }): Promise<Installation>;

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[banderror]

Hey @kpollich, here's the ticket we promised earlier. @xcrzx is going to switch to it next week (week of July 8). Could we please find someone to actively assist with it from the Fleet side (be available for questions, pair programming, code review, etc)?