Fine-tuning zstd's compression parameters for our use case.
Changes
Disable long-distance matching. Based on my benchmarks, long-distance matching improves compression ratio very little, but slows down compression significantly (especially on Linux)
Investigation details
I made the benchmarks on Bitrise VMs using zstd's benchmark mode and a tar of a typical Gradle build cache.
Checklist
step.yml
andREADME.md
is updated with the changes (if needed)Version
Requires no version update
Context
Fine-tuning zstd's compression parameters for our use case.
Changes
Disable long-distance matching. Based on my benchmarks, long-distance matching improves compression ratio very little, but slows down compression significantly (especially on Linux)
Investigation details
I made the benchmarks on Bitrise VMs using zstd's benchmark mode and a tar of a typical Gradle build cache.
Commands:
Results:
Numbers are: compression ratio, compression speed, decompression speed
Linux (Standard VM):
Linux uses an older version of zstd, the latest version has some speed gains, but
--long
is still a lot slower.macOS:
Decisions