aschampion / rust-n5

Rust implementation of the N5 "Not HDF5" tensor file system format
Apache License 2.0
15 stars 3 forks source link

WebAssembly support #10

Closed aschampion closed 6 years ago

aschampion commented 6 years ago

This can either be implemented as a feature flag or dependent crate.

Blocked on:

Because there may be issues with blocking on Futures in WASM, this may also depend on #9.

aschampion commented 6 years ago

See also zarr-developers/zarr#289

jakirkham commented 6 years ago

Saw n5-wasm popped up recently. Congrats! 😄

Curious to hear more about the plan with it. 😉

aschampion commented 6 years ago

Thanks! I haven't publicly plugged it anywhere because it still needs some cleanup and testing, but it works at the moment in the sense that if you have an N5 stack hosted via a plain HTTP file server, you can read attributes, blocks (into flat typed arrays of the appropriate type, e.g., Uint8Array), and such.

It only supports raw and gzip-compressed datasets for now, since the other compression backends aren't yet wasm-compatible (either wrap C libraries or use std/platform features not available in Rust's wasm32-unknown-unknown compilation target). In practice this isn't an issue for us, since all of our real datasets are gzip.

Our plan to get it into use is to add an image block rendering layer to CATMAID that uses n5-wasm to fetch blocks. Once we've used that experience to iron out the ergonomics, performance, and build a simple n-dim access layer, we'll publish the crate on crates.io and the wasm package to npm, which is trivial in Rust thanks to wasm-pack.

This will finally allow us to have orthoviews into stacks sharing their image cache.

Note that it's currently Chrome only because it relies on BigInt support. I know there are polyfills out there, but I don't have any motivating need to run on other browsers.

cc @tomka @axtimwalde