101arrowz / fflate

High performance (de)compression in an 8kB package
https://101arrowz.github.io/fflate
MIT License
2.27k stars 79 forks source link

Hangs on gunzipSync of compressed content with invalid dictionary #173

Closed wmertens closed 1 year ago

wmertens commented 1 year ago

I'm using fflate to add compression with a dictionary to https://qwik.builder.io/playground URLs, and when I'm messing with the dictionary to optimize it, windows that are open will try to parse the existing URL and that ends up in them hanging.

I'm using gunzipSync, see https://github.com/BuilderIO/qwik/pull/4839/files#diff-70381ad31e6812d765710d06929534b8d6811b54a9f22bcb179d34a2bb0b2d1cR153

Sadly, even appending to the dictionary causes this. Would it ever be possible to allow that?

Should I be using a different compression algorithm?

Many thanks for your excellent library!

wmertens commented 1 year ago

:facepalm: of course it was my code hanging on the now corrupt string.

I'll close this, I assume extending dictionaries will never be possible.

101arrowz commented 1 year ago

For future reference, dictionaries are just treated as a prefix to an existing data source. When you compress some source data with a dictionary, you can think of it as compressing the dictionary concatenated with the source and extracting the bytes that correspond to just the source. A corrupt dictionary provided during decompression can never cause fflate to throw an error, but will cause it to return invalid data.

And to your question, extending (adding a suffix to) a dictionary is basically guaranteed to corrupt it. You can add a prefix to a dictionary with no ill effect, but it won't change how decompression works.

wmertens commented 1 year ago

Oh a prefix, great idea 👍 Thanks!