Majored / rs-async-zip

An asynchronous ZIP archive reading/writing crate.
MIT License
135 stars 44 forks source link

A way to `write_entry_stream` while retaining valid local file headers #147

Open oeed opened 3 months ago

oeed commented 3 months ago

I am trying to generate ZIPs in a fairly low memory environment for files that can be quite large, so reading the entire contents and using write_entry_whole unfortunately isn't feasible. I have been using write_entry_stream, which does work well, but I've found Java based consumers reject files with invalid local headers with errors such as:

java.io.IOException: java.util.zip.DataFormatException: invalid stored block lengths

The docs do mention this:

The generated Local File Header will be invalid, with no compressed size, uncompressed size, and a null CRC. This might cause problems with the destination reader.

But I was wondering if there was some potential way around it, or a way to write chunks of data rather than in whole entries? Thank you for this crate!

Majored commented 2 months ago

It sounds like the Java reader doesn't support files with data descriptors then, unfortunately.

Does the writer you're using implement Seek? One option I've been meaning to implement is double-writing, where you write a dummy local file header as you're going along, and then seek back and write the correct values in the LFH (and don't use a data description - acting as if it was written with write_entry_whole). Though obviously, this will require a bit of work so is not an immediate solution.