search

WordPress / wordpress-playground

Run WordPress in the browser via WebAssembly PHP
https://w.org/playground/
GNU General Public License v2.0
1.64k stars 257 forks source link

Blueprints: Streaming zip/unzip for browser and server #482

Open eliot-akira opened 1 year ago

eliot-akira commented 1 year ago

Currently, the blueprint step unzip requires a path to the zip file. It would be useful to add support for passing a Stream.

It has advantages in terms of speed and memory use, in that it starts unzipping as data comes in instead of buffering it all in memory, writing it to a temporary file, then passing the path.

Another common use case is when the caller of unzip has a File instance, which cannot be passed as is. I suppose in this case it would be useful to be able to pass the File instance instead of creating a stream from it, or writing/copying it to the file system.

..didn't we pipe the zip file in another PR straight into unzip so we could skip the temporary file?

that happens in wp-now and cannot be easily done with Blueprints yet because the "resource" abstraction outputs a File object. Also, the zip stream was node-specific. Figuring this out would be a great follow-up, though.

Loosely thinking, maybe a "zipped resource type" or the ability to output a stream from a regular resource would do the trick.

A difficulty is that Stream is available only on Node.js. While a port to the browser exists (readable-stream), I'm guessing it doesn't do what we need, which is to stream data from JavaScript to PHP/WASM - and vice versa.

..If the File instance has the file content in memory on JS side, maybe it can be passed directly to WASM as a transferable object instead of copying it or writing to a temporary file, and stream it to PHP somehow.

adamziel commented 9 months ago

Related explorations:

adamziel commented 6 months ago

Switching to https://github.com/WordPress/blueprints-library/ will resolve this issue as it uses native PHP streams for all the file transfers.