Make decompressed data from decompress_to_vec accessible on failure

[RazrFalcon]

Running this code with this file (decompress zip first).

miniz_oxide::inflate::decompress_to_vec_zlib(input)

produces the FailedCannotMakeProgress error.

Based on the header, it seems to be a zlib-compressed data and I have a 3rd-party app that opens it successfully.

Is it a miniz_oxide bug or data is somehow malformed?

miniz_oxide: 0.5.1

[oyvindln]

From what I can see it's incompletely compressed data, it doesn't seem like the last block is indicated to be the final one. The simple decompression methods (decompress_to_vec_xx) do not have a way to recover data compressed so far if there is a failure at some point, you would need to use the lower level streaming decompression (optionally via flate2).

When testing other tools that handle raw zlib data, zlib-flate from qpdf dumps out about a 4.1 mb file (which is mostly zeroes but with a few FF bytes occasionally), while pigz gives pigz: skipping: data.zz: corrupted -- incomplete deflate data (which indicates Z_BUF_ERROR from zlib). Not sure what the file is supposed to be, it seems a bit odd in general.

[RazrFalcon]

Can you clarify how can I handle it correctly using low-level API?

Let's say I have something like:

fn decode(input: &[u8], buf: &mut [u8]) -> Result<(), Error> {
    use miniz_oxide::inflate::core::inflate_flags;

    let flags =
          inflate_flags::TINFL_FLAG_PARSE_ZLIB_HEADER
        | inflate_flags::TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF;

    let mut decomp = Box::<miniz_oxide::inflate::core::DecompressorOxide>::default();

    let (status, _, _) = miniz_oxide::inflate::core::decompress(&mut decomp, input, buf, 0, flags);
    dbg!(status); // FailedCannotMakeProgress
    Ok(())
}

How can I recover all the data using this code? Or should I use flate2 to begin with?

Sadly, I cannot clarify what this data is from. It's an image, so having a lot of zero bytes is fine. All I can say that this data is valid (in the application sense, and not zlib one) and there are multiple proprietary apps that handle it perfectly. Therefore I'm trying to understand how can I process it too.

[RazrFalcon]

Hi again. I've tried using flate2 but it also fails. Any suggestion about how it can be solved?

[RazrFalcon]

flate2 with zlib backend also fails.

[RazrFalcon]

It seems like in this particular case the bytes_written (the last value in decompress() return tuple) is exactly the same as I need. Meaning the decoder read all the data I want and I can simply ignore FailedCannotMakeProgress error.

Feel free to close this issue if this is not something that can be fixed on the library side.

[oyvindln]

Sorry completely forgot about this one, but yeah, if you use e.g flate2 or the low level streaming decompression the workaround is to decode until there is no more data to input and you have extracted all the currently decompressed data and then ignore the error.

[oyvindln]

Outputting the data that has been decompressed is something that could be changed for this function, though it would be an API break, so would have to wait until a future version if it's relevant.

[oyvindln]

Can add this together with making alloc optional in the next release since that needs a version bump anyhow.

[oyvindln]

Fixed in 8179633092b032b783333c73395490b6b9243823