search

gildas-lormeau / zip.js

JavaScript library to zip and unzip files supporting multi-core compression, compression streams, zip64, split files and encryption.
https://gildas-lormeau.github.io/zip.js
BSD 3-Clause "New" or "Revised" License
3.38k stars 510 forks source link

Use webworker for stream copy #364

Closed peaceps closed 1 year ago

peaceps commented 1 year ago

In zipjs I find that for not compressed entry it will never use webworker. While in my case I also want to use webworker to do the copy. Is it possible to add an option to support this? Thanks.

gildas-lormeau commented 1 year ago

Why do you want to use a web worker in such a case?

peaceps commented 1 year ago

In my zip file there're lots of zip/gz/tar files inside, they will be copied in chrome main thread, which will cost much resource of main thread and will also be affected by user action(minimize chrome/switch to other tab)

gildas-lormeau commented 1 year ago

The problem is that web workers would do nothing. It would just help to slow down the extraction (thanks to message passing).

In zip.js, the main thread is always responsible for concatenating chunks. A way to solve this issue would be to pass the ReadableStream and the WritableStream to the web worker when extracting an entry from a zip file. Thus the data would be concatenated in the worker instead of the main thread. This requires some refactoring but it should be doable.

gildas-lormeau commented 1 year ago

I just pushed the version 2.6.24 which allows you to pass a new option when creating a new ZipReader instance or when calling ZipReader#getData(). This new option is named transferStreams and is set to true by default. When set to false, zip.js should work as before. When set to true, zip.js will try to transfer the ReadableStream and the WritableStream instances to the web workers. Thus, the decompression is entirely done in the web workers. Note that currently this feature is not supported in Deno.

peaceps commented 1 year ago

Thanks so much!