search

richgel999 / miniz

miniz: Single C source file zlib-replacement library, originally from code.google.com/p/miniz
MIT License
2.21k stars 323 forks source link

busybox's unzip cannot read miniz-produced zips because of streaming flag #180

Open vadimkantorov opened 3 years ago

vadimkantorov commented 3 years ago

I implemented a simple zip-like utility based on miniz: https://github.com/vadimkantorov/nanozip/blob/master/nanozip.c (roughly a sequence of mz_zip_writer_init_file, mz_zip_writer_add_mem, mz_zip_writer_add_file, mz_zip_writer_finalize_archive, mz_zip_writer_end).

Unfortunately, busybox's restricted unzip cannot decompress miniz-produced zips and fails with unzip: zip flag 8 (streaming) is not supported at https://git.busybox.net/busybox/tree/archival/unzip.c#n743

Would you have any advice? Is it possible to ask miniz to not use this flag?

Thank you!

bubnikv commented 3 years ago

miniz writes the compressed and uncompressed file size after the data, for that the streaming flag is required. If the streaming flag was removed, miniz would have to update these sizes in the file header before the file data after the compressed file size is known, that is after finishing the data.

uroni commented 2 years ago

Is this issue fixed by setting the new MZ_ZIP_FLAG_WRITE_HEADER_SET_SIZE flag?

vadimkantorov commented 2 months ago

It looks like this new flag should help (but then it also should unset the streaming flag). Too bad I did not conserve failing miniz-produced files examples :(

At I found two examples at https://github.com/fadams/serverless/blob/master/hybrid_fn_openfaas_stack/examples/fntest/bsdtar/README.md#streaming-unzip:

It appears that busybox falls into this branch when CDF could not be detected via this function: https://git.busybox.net/busybox/tree/archival/unzip.c#n233 - only then the streaming flag is the problem. So maybe somehow miniz also produced streaming archives which had tricky CDF/CDE placement incompatible with this busybox CDF detection algo? busybox says that normally this code path is taken if the file is not seekable (e.g. streaming over network), but this was not the case with the files I had produced with miniz...