search

image-rs / deflate-rs

An implementation of a DEFLATE encoder in rust
Apache License 2.0
53 stars 14 forks source link

Add support for specifying zlib windowBits #45

Closed camden-smallwood closed 4 years ago

camden-smallwood commented 4 years ago

This adds support for specifying zlib windowBits parameter in CompressionOptions via the with_window_bits function.

Example using default compression and windowBits of 10:

let deflated_data = deflate::deflate_bytes_zlib_conf(data,
    deflate::CompressionOptions::default().with_window_bits(10)
);
oyvindln commented 4 years ago

Completely missed this PR. The windowbits flag indicates how large of a buffer of previous data the compressor used when compressing the data. There isn't any support for restricting it in the library at the moment (other than RLE encoding which is technically a buffer of length one.) It's not a parameter that makes much sense to specify manually as implemented in this PR unless the goal is to confuse the decompressor.

camden-smallwood commented 4 years ago

Understandable, I just have a specific case that required me to use specific window bits (10 in this case). When I tried to zlib deflate the data using the crate as it is, the target application would crash, pointing to invalid compressed data. After adding the changes to allow explicit window bits, the application stopped crashing and the data worked as expected. If there's a different approach you could recommend, I would appreciate it, but I couldn't find anything that would accomplish the same end result after a week of digging.

oyvindln commented 4 years ago

That seems like a very odd handling of the parameter by the application. If it failed due to windowbits being set too low, it would make sense, but crashing when windowbits is higher (even though it's compressed with a full buffer) seems rather odd.

What I would suggest for now if possible is to try to use flate2 and use the zlib back-end as it allows you to set the windowbits (which unlike in this PR actually sets the buffer size properly when compressing) manually.

Alternatively you could manually change the header bytes after compressing, or use a raw deflate stream and add the header + checksum manually afterwards.