search

mozilla / sccache

Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.
Apache License 2.0
5.85k stars 552 forks source link

Add custom zstd compression level #2199

Closed nyanrus closed 4 months ago

nyanrus commented 5 months ago

fix #2187 Allow custom zstd cache compression level

sylvestre commented 5 months ago

Could you please add a test and mention this new variable in the doc? thanks

codecov-commenter commented 5 months ago

Codecov Report

Attention: Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Project coverage is 42.80%. Comparing base (0cc0c62) to head (8caeebb). Report is 63 commits behind head on main.

Files Patch % Lines
src/cache/cache.rs 66.66% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2199 +/- ## =========================================== + Coverage 30.91% 42.80% +11.89% =========================================== Files 53 53 Lines 20112 20484 +372 Branches 9755 9685 -70 =========================================== + Hits 6217 8768 +2551 - Misses 7922 8075 +153 + Partials 5973 3641 -2332 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

glandium commented 5 months ago

This should alter the hash key, too.

nyanrus commented 5 months ago

Thank you for reply.

This should alter the hash key, too.

This is just cache compression level and don't affect to decompression mechanism. Is there need to change hash key? or if your intend is "It should not alter hash key, but will changed by this code," how I except this env in hash generation? thanks

nyanrus commented 5 months ago

Thank you for reply.

Could you please add a test

It seems no tests for SCCACHE_DIR and SCCACHE_CACHE_SIZE that is basic feature like compression level. What I should prove in the test?

and mention this new variable in the doc

I'll add this to docs. Is this okay adding to Local.md and Configuration.md?

glandium commented 5 months ago

This is just cache compression level and don't affect to decompression mechanism. Is there need to change hash key?

I would say there is. If you're setting the variable for greater compression, you're probably expecting your cache to be more greatly compressed, not to continue containing whatever it contains right now. Of course, the caveat is that those lesser compressed objects will stay in the cache too, until the cache is full and they are pushed out as being the least recently used.

nyanrus commented 5 months ago

Thank you for reply!

you're probably expecting your cache to be more greatly compressed

But when the hash changes, some problem can occur. You cannot acquire less build time without same compression level setting. And as you told me, when you are using unlimited storage like local cache, it will take more size by unused cache, or you should wait until it is pushed out in limited storage.

Rather than that, just adding a warning as "already compressed cache are not replaced" seems reasonable. Because it will take the same time to make new compressed caches in both cases, changing hash and not. And adding some note can help to understand how the option work, like "you should make a new cache."

sylvestre commented 5 months ago

could you please add a test in the integration testsuite ? https://github.com/mozilla/sccache/blob/main/.github/workflows/integration-tests.yml thanks

nyanrus commented 4 months ago

@sylvestre is the test enough?