search

Ne-Lexa / php-zip

PhpZip is a php-library for extended work with ZIP-archives.
MIT License
491 stars 60 forks source link

Avoid memory with outputAsAttachment + instant download #46

Open jeanpaze opened 4 years ago

jeanpaze commented 4 years ago

First, thank you @Ne-Lexa for this amazing library!

I'm trying to implement your library on my Nextcloud server (running as a Docker container on my Synology NAS) but when I trigger your lib, it takes too much time to actually start the download and I ended up by having a 502 bad gateway (timeout) session from nginx, especially with big files (>4GB). In my case, it's not possible to change that timeout because of Cloudflare proxy (free account).

I want to use your lib mostly because of the password feature that is not available on Nextcloud's PHP ZipStreamer but I'm trying to make it work with the basics first.

For context, Nextcloud uses this lib and all the main calls comes from here.

Somehow, the zip download starts instantly with ZipStreamer.

I managed to set up everything correctly using addFromStream method, and this is what I tried already. Grouping the code here:

$this->streamerInstance = new ZipFile();
$this->streamerInstance->addFromStream($stream, $internalName);
$this->streamerInstance->outputAsAttachment($this->streamFilename, 'application/zip');
$this->streamerInstance->close();

I also make sure that the stream from each file is not being close (the opposite from ZipStreamer).

I tried to change from php://temp to php://output inside outputAsAttachment method but it was not enough - seems like it was missing the mime type.

Is it possible to, somehow avoid the memory and download/stream instantly the zip by blocks, as I think it is from ZipStreamer?

Log:

Allowed memory size of 1048576000 bytes exhausted (tried to allocate 3177488512 bytes) at /nelexa/zip/src/ZipFile.php

Please let me know if it's clear enough or if you need more info. Thanks!

Ne-Lexa commented 4 years ago

Hello.

  1. What version are you using? Upgrade to ^ 3.3. This version fixes a memory issue.
  2. Files with a large size should not be created in a web server session. Use cli or queues.
  3. Try the solution from this answer - https://github.com/Ne-Lexa/php-zip/issues/45#issuecomment-588132545. It should be best suited to your needs.
jeanpaze commented 4 years ago

1 - I'm using 3.3.0 2 - Understandable but I still have the same issue with 7zip cli - 502 bad gateway. How about queues? Do you have any example? 3 - Yes, I tried exactly that command

Did you get the chance to take a look at ZipSteamer lib?