search

httptoolkit / brotli-wasm

A reliable compressor and decompressor for Brotli, supporting node & browsers via wasm
Apache License 2.0
264 stars 21 forks source link

Streaming support #10

Closed myl7 closed 2 years ago

myl7 commented 2 years ago

Currently WASM to/from JS of brotli-wasm uses UInt8Array to pass data, which requires all data loaded into memory in advance. It is OK for small files (TBH it is common for Brotli) but as for big files such as > 1G, this will cause a huge memory usage.

brotli crate provides steaming ways such as Compressor writer to (de)compress data piece by piece. It would be nice for the package to include them.

I am the maintainer of brotli-dec-wasm and a streaming binding implementation is included in it. If the feature is recognized as required, I would be willing to giving a PR to implement the feature for the repo.

pimterry commented 2 years ago

Sure, that'd be great! If you have time I'm definitely open to a PR to add this.