search

EngFlow / bazel_invocation_analyzer

A tool that automatically analyzes a Bazel build's profile and offers suggestions on how to make it faster.
https://analyzer.engflow.com
Apache License 2.0
90 stars 10 forks source link

Reduce memory requirements of running tool #163

Closed saraadams closed 10 months ago

saraadams commented 10 months ago

Problem

Especially when analyzing larger Bazel profiles, the tool requires a lot of memory. This may lead to OOMs.

Preferred solution

Investigate what consumes the most memory and look into implementing it more performantly.

Additional information

When analyzing a Bazel profile that is gzipped 34 MB and unzipped 35 MB, running the analyzer consumed about 4 GB.

saraadams commented 10 months ago

The main culprit seems to be how we read the Bazel profile. We read the whole profile as a JsonObject. Instead, we should consider reading it in a stream.