search

kbrw / zip_stream

Library to read zip file in a stream. Zip file binary stream -> stream of {:new_file,name} or uncompressed_bin Erlang zlib library only allows deflate decompress stream. But Erlang zip library does not allow content streaming.
7 stars 5 forks source link

Zipping files from streams #1

Open Lazarus404 opened 8 years ago

Lazarus404 commented 8 years ago

I've noticed this library also contains a zip function, but no examples of how to use it. I need to download a bunch of files from AWS S3, zip them as they download, then feed the zip to the requesting user via Phoenix response. Is this possible with this library?

Thanks

awetzel commented 8 years ago

Unfortunatly, the ZIP Spec defines a way to allow streaming when writing the file with a "data descriptor" at the end of each file defining size and checksum AFTER the file. But nearly none implementation understand this data descriptor. So no this lib will not create a zip as stream since it is not actually possible :)

But I plan to add support to write zip to a file with an Elixir binary "stream" input.

Lazarus404 commented 8 years ago

Hi Arnaud,

This is what I'm attempting to do in Elixir: http://engineroom.teamwork.com/how-to-securely-provide-a-zip-download-of-a-s3-file-bundle/

I don't know if that library helps? I'm essentially hoping to reduce memory and drive requirement on the server

Regards, Lee

On Mon, May 9, 2016 at 6:15 AM, Arnaud Wetzel notifications@github.com wrote:

Unfortunatly, the ZIP Spec defines a way to allow streaming when writing the file with a "data descriptor" at the end of each file defining size and checksum AFTER the file. But nearly none implementation understand this data descriptor. So no this lib will not write zip file as stream since it is not actually possible :)

But I plan to add support to write zip to a file with an Elixir binary "stream" input.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/awetzel/zip_stream/issues/1#issuecomment-217737331