search

allanlei / python-zipstream

Like Python's ZipFile module, except it works as a generator that provides the file in many small chunks.
GNU General Public License v3.0
128 stars 34 forks source link

Support Zip64 when compressing iterables and strings #25

Open arjan-s opened 7 years ago

arjan-s commented 7 years ago

Zip64 support currently doesn't work when the inputs for the generated zip file are iterables and/or strings. The reason for this is that __write() assumes a file size of 0 bytes for those inputs. Because of the file size the module doesn't enable Zip64 support. When it detects during compression that the file size is larger than ZIP64_LIMIT, it (correctly) raises RuntimeError('File size has increased during compressing').

This patch adds the optional (and thus backwards compatible) argument buffer_size to write_iter and writestr. This allows programs using the module to specify the buffer size that will result from the iterable or string, and in turn that allows __write() to enable Zip64 support when necessary.

matthewatabet commented 5 years ago

Would it be possible to merge this? I ran into this issue because I need to stream very large amounts of data.

arjan-s commented 5 years ago

Hi @matthewatabet, it seems the original developer of this package is not active anymore. So I've forked the package as zipstream-new: https://github.com/allanlei/python-zipstream/issues/33