Add cache hit percentage to stats

[Saruniks]

Closes #2161

Inspiration from #614

Feedback is welcome.

---

No compile requests:

Compile requests                      0
Compile requests executed             0
Cache hits                            0
Cache misses                          0
Cache hits rate                       -
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average compiler                  0.000 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/home/ubuntu-user/.cache/sccache"
Use direct/preprocessor mode?   yes
Version (client)                0.8.1
Max cache size                       10 GiB

Zero cache hits:

Compile requests                    397
Compile requests executed           319
Cache hits                            0
Cache misses                        316
Cache misses (C/C++)                 52
Cache misses (Rust)                 264
Cache hits rate                    0.00 %
Cache hits rate (C/C++)            0.00 %
Cache hits rate (Rust)             0.00 %
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  3
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                  73
Non-compilation calls                 5
Unsupported compiler calls            0
Average cache write               0.004 s
Average compiler                  0.370 s
Average cache read hit            0.000 s
Failed distributed compilations       0

Non-cacheable reasons:
crate-type                           50
unknown source language              18
-                                     3
-E                                    1
incremental                           1

Cache location                  Local disk: "/home/ubuntu-user/.cache/sccache"
Use direct/preprocessor mode?   yes
Version (client)                0.8.1
Cache size                          234 MiB
Max cache size                       10 GiB

Some cache hits:

Compile requests                   1189
Compile requests executed           957
Cache hits                          632
Cache hits (C/C++)                  104
Cache hits (Rust)                   528
Cache misses                        316
Cache misses (C/C++)                 52
Cache misses (Rust)                 264
Cache hits rate                   66.67 %
Cache hits rate (C/C++)           66.67 %
Cache hits rate (Rust)            66.67 %
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  9
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                 217
Non-compilation calls                15
Unsupported compiler calls            0
Average cache write               0.004 s
Average compiler                  0.370 s
Average cache read hit            0.004 s
Failed distributed compilations       0

Non-cacheable reasons:
crate-type                          150
unknown source language              54
-                                     7
incremental                           3
-E                                    3

Cache location                  Local disk: "/home/ubuntu-user/.cache/sccache"
Use direct/preprocessor mode?   yes
Version (client)                0.8.1
Cache size                          234 MiB
Max cache size                       10 GiB

sccache --show-adv-stats:

Compile requests                     401
Compile requests executed            319
Cache hits                           316
Cache hits (c [gcc])                  52
Cache hits (rust)                    264
Cache misses                           0
Cache hits rate                   100.00 %
Cache hits rate (c [gcc])         100.00 %
Cache hits rate (rust)            100.00 %
Cache timeouts                         0
Cache read errors                      0
Forced recaches                        0
Cache write errors                     0
Compilation failures                   3
Cache errors                           0
Non-cacheable compilations             0
Non-cacheable calls                   77
Non-compilation calls                  5
Unsupported compiler calls             0
Average cache write                0.000 s
Average compiler                   0.000 s
Average cache read hit             0.002 s
Failed distributed compilations        0

Non-cacheable reasons:
crate-type                            51
unknown source language               18
-                                      5
incremental                            2
-E                                     1

Cache location                  Local disk: "/home/ubuntu-user/.cache/sccache"
Use direct/preprocessor mode?   yes
Version (client)                0.8.1
Cache size                           235 MiB
Max cache size                        10 GiB

[AJIOB]

Hi @Xuanwo,

Is it OK to you?

[sylvestre]

Could you please add a test to verify the field is present? Thanks

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 65.78947% with 91 lines in your changes missing coverage. Please review.

Project coverage is 40.88%. Comparing base (0cc0c62) to head (1234cef). Report is 64 commits behind head on main.

Files	Patch %	Lines
src/server.rs	45.31%	18 Missing and 52 partials :warning:
tests/helpers/mod.rs	78.26%	10 Missing and 5 partials :warning:
tests/cache_hit_rate.rs	91.30%	6 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #2211 +/- ## ========================================== + Coverage 30.91% 40.88% +9.97% ========================================== Files 53 55 +2 Lines 20112 20673 +561 Branches 9755 9794 +39 ========================================== + Hits 6217 8453 +2236 - Misses 7922 8091 +169 + Partials 5973 4129 -1844 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[AJIOB]

Hi @Saruniks

Is it ready to be merged?

[Saruniks]

Hello @AJIOB

Ready from my side. Unless perhaps someone has comments on the code and tests.

[AJIOB]

Hi @sylvestre & @Xuanwo Can we merge it?

[sylvestre]

i was hoping to have also integration tests like in https://github.com/mozilla/sccache/blob/main/.github/workflows/integration-tests.yml and a verification that the json files contain the fields too

[sylvestre]

and sorry for the latency

[Saruniks]

Ok. I'll add the integration test

[sylvestre]

many thanks :)

[Saruniks]

Hello,

a) I added integration tests. Not to the GitHub Actions workflow like mentioned above, but "Rust" ones (see tests/cache_hit_rate.rs).

I could add extra jobs to the GitHub Actions workflow, no problems. But I thought that what I developed could be sufficient as no integration with external tool needs to be tested. Let me know if you think otherwise.

b) Regarding the json stats: Apologies for not mentioning it before, but I didn't develop cache hit rate display for json stats.

The problem is that json stats is just serialized structs. When "StatsFormat::Text" stats are achieved by using "complex" print method.

    match fmt {
        StatsFormat::Text => stats.print(advanced),
        StatsFormat::Json => serde_json::to_writer(&mut io::stdout(), &stats)?,
    }

I am still not sure what would be the best way to implement cache hit rate for json stats. Maybe custom serialize implementation would work (and not derive trait like currently), but then it could be considered magic-like. But then calculating cache hit rate on every stats change could be considered too wasteful (or considered as calculated value anti-pattern).

So not sure how to proceed with json stats.

[sylvestre]

don't bother for the json output thanks!

[sylvestre]

I guess you saw that some tests are failing

[Saruniks]

I fixed the issue with failing tests. Checks should pass now.