gildas-lormeau / zip.js

JavaScript library to zip and unzip files supporting multi-core compression, compression streams, zip64, split files and encryption.
https://gildas-lormeau.github.io/zip.js
BSD 3-Clause "New" or "Revised" License
3.38k stars 510 forks source link

Using this with lots of data on iOS Safari crashes website #391

Closed schlagmichdoch closed 1 year ago

schlagmichdoch commented 1 year ago

First of all: Thanks a lot for this awesome, lightweight library!

I implemented it similarily to the create file demo and got an issue that is reproducable on the demo as well: https://gildas-lormeau.github.io/zip.js/demos/demo-create-file.html

Using iOS Safari:

  1. Add more than 208 MB of files: Everything works fine: All Files are progressed successfully
  2. Click "Download" Website crashes: A reload is forced after a blue banner on the top of the page appears for half a second that reads: "A problem occurred with this webpage so it was reloaded"

I guess it has to do with the maximum RAM a browser can occupy which is 384 MB on iOS Safari 15. Do you know of any workaround to that problem? Would it help to use the SplitDataReader class? I would appreciate any help!

gildas-lormeau commented 1 year ago

I don't have an iOS device and I was not able to reproduce the issue in the simulator bundled with Xcode.

I guess the Origin Private File System API could help, see https://webkit.org/blog/12257/the-file-system-access-api-with-origin-private-file-system/. However it's a bit tricky to use because FileSystemFileHandle#createSyncAccessHandle is only available in a Worker. The SplitDataReader would not be useful. However you could create a split zip with by using a generator ofWriter, it could help to save memory. There's a test showing how to use it here: https://github.com/gildas-lormeau/zip.js/blob/910cdf666382d5738a69c490b79483ce4b96df28/tests/all/test-split-zip.js and a similar demo here: https://plnkr.co/edit/SkTNcrTq8XL37mQo?preview.

schlagmichdoch commented 1 year ago

Thanks for the quick response!

So your idea is to create a zip file on the device itself via the Private File System API and only upload it then? Not sure whether that would reduce memory usage though.

I decided to remove zip.js on iOS completely and go for another approach so the browser does not have to save the same blob twice. As I have the same problem when using navigator.share with big files though, I simply won't allow files bigger than 200 MB on iOS.

If you're interested where your project got implemented into: Peer to peer file sharing https://pairdrop.net/ (Repo)

As this issue is not really zip.js specific I'll close it, thanks for helping though!

gildas-lormeau commented 1 year ago

I did not know that you needed to upload the data. In this case, you just needed to write a Writer class that would have done so... It's approx. 20 lines of code. Anyway, since you decided to not use zip.js, I won't bother you.