search

101arrowz / fflate

High performance (de)compression in an 8kB package
https://101arrowz.github.io/fflate
MIT License
2.27k stars 79 forks source link

Option to shorten output as much as possible #174

Closed wmertens closed 1 year ago

wmertens commented 1 year ago

When using the compressed data within an application, you know what the data is and what it's compressed with, so it would be nice to strip unnecessary preamble, like the magic bytes and timestamp, and maybe some of the suffix? I'd want to keep the crc check though.

101arrowz commented 1 year ago

Assuming this is for GZIP, fflate produces a minimal output compliant with the specification. There is no way to shorten it without producing invalid GZIP data. If you want a format with no headers, you can just use raw DEFLATE (deflateSync and inflateSync). You could probably append your own checksum to the end of that data if you want some error checking.

Please note that the CRC built into GZIP isn't actually verified during decompression by fflate because in practice, any bit/byte errors in a compressed stream will actually create an invalid data stream that throws an error anyway, with very few exceptions. (The CRC is created properly during compression of course).

wmertens commented 1 year ago

Many thanks for your help, this makes a lot of sense, my URLs are super short now 🤩

Indeed, even if it manages to decrypt a corrupted input, the resulting data also needs to be parseable. I'll take my chances without a checksum :)

wmertens commented 1 year ago

Thanks again for your help, you can see the compression in action at https://qwik.builder.io/playground if you like :)