search

EngFlow / bazel_invocation_analyzer

A tool that automatically analyzes a Bazel build's profile and offers suggestions on how to make it faster.
https://analyzer.engflow.com
Apache License 2.0
90 stars 10 forks source link

Duplicate collection of cache checks and misses #156

Closed saraadams closed 11 months ago

saraadams commented 11 months ago

Currently two data providers collect statistics on how often a remote cache was checked, and how many were hits/misses. One of the two can be dropped. CachingAndExecutionMetrics (new) seems to do a better job at it than RemoteCacheMetrics.

Example profile collected from a fully incremental build using --disk_cache:

Bazel output: INFO: 3869 processes: 3541 disk cache hit, 325 internal, 2 darwin-sandbox, 1 local.

BIA:

CachingAndExecutionMetrics: Metrics on the processed actions' caching and execution status.
Actions: ____________________________ 3869
    Remote cache hits: ______________ 3541     91,52% of all actions
    Remote cache misses: ____________    0      0,00% of all actions
        Executed locally:                0      0,00% of all cache misses
        Executed remotely:               0      0,00% of all cache misses
        Not reported:                    0      0,00% of all cache misses
    Remote cache not checked: _______  302      7,81% of all actions
        Executed locally:                3      0,99% of all cache skips
        Executed remotely:               0      0,00% of all cache skips
        Internal (not exhaustive):      26      8,61% of all cache skips
        Not reported                   299     99,01% of all cache skips

    Executed locally: _______________    3      0,08% of all actions
    Executed remotely: ______________    0      0,00% of all actions
    Execution not reported: _________ 3866     99,92% of all actions

RemoteCacheMetrics: Metrics on the remote caching used. This includes both the use of `--remote_cache` and `--disk_cache`.
Number of cache checks:               3540
Number of cache misses:               2313
Cache hit percentage:               34,66%
Time spend checking for cache hits: 11s
Time spend downloading outputs:     209ms
Time spend uploading outputs:       0ms

CachingAndExecutionMetrics matches with the Bazel output on disk cache hit, so we should remove the cache check evaluation from RemoteCacheMetrics and keep the data collected by CachingAndExecutionMetrics