search

Majored / rs-async-zip

An asynchronous ZIP archive reading/writing crate.
MIT License
123 stars 40 forks source link

Streaming - Write file in chunks #125

Closed inzanez closed 2 months ago

inzanez commented 3 months ago

Hi

it seems that writing a file in chunks is not possible, is that correct? As far as I can see, write_entry_stream would support writing an entry in a streaming fashion but generated files are not really 'unzipable' anymore.

Looking at the ZIP structure, shouldn't it be possible to fill the correct data into the central directory located at the end of the ZIP file,...allowing to get all the information required to unzip that given file?

Majored commented 3 months ago

shouldn't it be possible to fill the correct data into the central directory located at the end of the ZIP file,...allowing to get all the information required to unzip that given file?

We already do this and is the expected behaviour as far as I can tell from the specification (it's stated that only the local file header values are ever zeroed when using a data descriptor, not the central directory record values). So in practice, there's very little difference between streamed vs non-streamed writes.

Though, when an entry is stream written and then stream read, this crate has a limitation in that the entry cannot use the Stored compression mode (ie. no compression used). I suspect this is a limitation of other libraries as well since this is caused by a fundamental conflict.

https://github.com/Majored/rs-async-zip/blob/eb5a43ed34438edbc5f763a5c59c28cb99525e8c/src/base/read/stream.rs#L21

inzanez commented 3 months ago

Ok, this was an error on my end. I forgot to call close on the ZIP file which resulted in a non-finished ZIP file. 7z was still able to read parts of it and therefore it seemed to contain one single huge file.