search

101arrowz / fflate

High performance (de)compression in an 8kB package
https://101arrowz.github.io/fflate
MIT License
2.27k stars 79 forks source link

Support adding files larger than 4GB #81

Closed jhiesey closed 3 years ago

jhiesey commented 3 years ago

What can't you do right now? As documented in the changelog, zip files cannot contain files greater than 4GB. It would be great if we could create zip files containing larger files, since it looks like the zip64 format can support that use case.

An optimal solution Ideally, the current streaming interface would automatically generate valid zip output when adding larger than 4GB files. It may also be necessary for the user to pass an option indicating that we need a zip64 header when adding a large file; that would be fine too.

As an interim solution, instead of adding full zip64 support, it would be nice if ondata returned an error when trying to write too large of a file instead of silently generating a corrupt .zip file. Currently, it looks like the generated .zip is the right size, but when unzipped it outputs truncated files. I suspect the size field just wraps around at 4GB; could you detect that and return an error?

101arrowz commented 3 years ago

Thanks for the feature request! I'll try to add support for Zip64 archives in the near future.

101arrowz commented 3 years ago

5 months later, I still haven't gotten time to do this. But after further consideration I'm recommending anyone who wants to handle files this big to use non-compressing, streaming ZIP generators like client-zip. The time it would take to compress a 4GB file, especially in JavaScript, is way too long to be practical for 99.9% of use cases, so a non-compressing ZIP generator is fine.