thegenemyers
commented
4 years ago One important feature to a lot of our users is that gunzip works on the compressed file. Is that true of a file compressed with libdeflate? If I understand correctly it would not be so as the compression format is different (or is it?). -- Gene
On 11/19/19, 11:35 AM, James Bonfield wrote:
If you are wedded to using Deflate, don't use Zlib as it's simply ancient technology. I'd advise libdeflate https://github.com/ebiggers/libdeflate/ instead as generally it's over double the performance and produces compatible data streams. It also offers (at a CPU cost) higher compression levels than zlib if desired.
However better still IMO given this is a new proposal is to use Zstd https://github.com/facebook/zstd/ instead. It's a better format than Deflate offering faster compression, decompression while being generally smaller. Basically it's a win-win-win.
(Better in terms of ratio is libbsc, but it has higher CPU so that's definitely a tradeoff and may not be approproate.)
For comparisons, see https://quixdb.github.io/squash-benchmark/unstable/ which shows the Pareto frontier. Obviously esoteric tools aren't appropriate, but it permits us to see how the standard well supported tools stack up against each other. Zstd covers quite a lot of the speed vs size tradeoffs optimally.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/VGP/vgp-tools/issues/1?email_source=notifications&email_token=ABUSINR5QHKUAPD7TA33GVTQUO6Q3A5CNFSM4JPBIMF2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H2JU6FA, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUSINQ6WYMFSINWFUILDWTQUO6Q3ANCNFSM4JPBIMFQ.
jkbonfield
If you are wedded to using Deflate, don't use Zlib as it's simply ancient technology. I'd advise libdeflate instead as generally it's over double the performance and produces compatible data streams. It also offers (at a CPU cost) higher compression levels than zlib if desired.
However better still IMO given this is a new proposal is to use Zstd instead. It's a better format than Deflate offering faster compression, decompression while being generally smaller. Basically it's a win-win-win.
(Better in terms of ratio is libbsc, but it has higher CPU so that's definitely a tradeoff and may not be approproate.)
For comparisons, see https://quixdb.github.io/squash-benchmark/unstable/ which shows the Pareto frontier. Obviously esoteric tools aren't appropriate, but it permits us to see how the standard well supported tools stack up against each other. Zstd covers quite a lot of the speed vs size tradeoffs optimally.