search

saghul / txiki.js

A tiny JavaScript runtime
MIT License
2.42k stars 162 forks source link

Support for compression/decompression library #545

Open KaruroChori opened 1 month ago

KaruroChori commented 1 month ago

At this point in time, there is no support for compression/decompression in txiki for what I can tell.
Users can bundle some like pure JS implementation like pako, but since quickjs is not exactly fast, that has a huge impact on performance. Binding zlib directly would be much more preferable from my point of view.

I am probably going to add it as a module myself, but even if the external module proposal is not mainlined I think there is a good case for some compression library to be shipped with txiki.

saghul commented 1 month ago

Yes! I've been thinking about wrapping libarchive. The annoying part is that it doesn't seem to be thread-safe, which means we'd need to run operations serialized in thre thread-pool, and watch out so no 2 things can run in parallel.

KaruroChori commented 1 month ago

Just my two cents on the subject. From what I can tell, libarchive is quite massive and with a very broad scope. It would make perfect sense as an external module, but for many it would just end up being mostly "bloat" as part of the default distribution. Something like zlib has a much more narrow focus, it self-describes as thread-safe and comes with all the basic features to support compression/decompression when implementing custom protocols (even just compressing HTTP) or data formats.

saghul commented 1 month ago

Good point.

There is already fflate bundled in, because it's a dependency of the ConpressionStream polyfill. Perhaps we could just expose it.

The reason I thought about libarchive is that in order to implement "tjs bundle" I need to download esbuild which comes in tar.gz form, so I'd need tar support too.

Alternatively I could just re-pack it and that's it...

KaruroChori commented 1 month ago

I would have preferred a native implementation, but that is specific for my use case. For general purpose I think exposing fflate would be best.

I was just wondering, esbuild has a wasm version: https://www.npmjs.com/package/esbuild-wasm So in theory we can just use it, with the advantage of being portable regardless of the architecture.
Aside from the untarring process, for which we can surely find a workaround.

saghul commented 1 month ago

I thought of that but wasm3 is not very fast and I'm afraid we might not have all the APIs necessary for it to work, but I also haven't tried it yet.

KaruroChori commented 1 month ago

I am just testing it, it does not work because of URL.createObjectURL missing (and potentially more)

saghul commented 1 month ago

The hanks for giving it a try!