search

ssbc / ssb-validate2-rsjs

Cryptographic validation of Scuttlebutt messages in the form of bindings (Rust) for Node.js.
3 stars 0 forks source link

Compile to WebAssembly #4

Closed mycognosist closed 3 years ago

mycognosist commented 3 years ago

Leaving this as a prompt for further investigation in the future. Relevant link: Compiling from Rust to WebAssembly

<@staltz> I was thinking if ssb-validate (sunrise choir) could be compiled to webassembly, since it doesn't have I/O, it's just computation, then it could run in ssb-browser-core?

<@arj03> yeah was thinking the same. Wondering how the threading works

mycognosist commented 3 years ago

Seems like a good time to post some updated notes on this process:

We are using wasm-bindgen to generate bindings from our Rust code. This is primarily used via the #[wasm_bindgen] attribute (used to mark functions for export and assign a JS function name) which is used in much the same way as the #[node_bindgen] attribute in our src/lib.rs for native.

Then we use the wasm-pack tool to "generate an npm wasm pkg from a rustwasm crate". wasm-pack uses wasm_bindgen under the hood.

Because our validation code relies on ssb-validate (rs) - which employs parallelization with rayon) - we use wasm-bindgen-rayon to auto-generate WebWorker shim code. These WebWorkers act as threads, with one WebWorker spawned for each thread invoked by the parallel validation code.

We are also using module-workers-polyfill to add support for Firefox (more context can be found in this issue)

At this point, the WASM code for ssb-validate2-rsjs is working, but only due to some convenient bugs 🙃 Additional context here.

I am now working to move the WASM initialization and WebWorker threadpool initialization off the main thread. Contextual details appear in this issue comment.

staltz commented 3 years ago

@mycognosist I guess we can close this one? :)

mycognosist commented 3 years ago

@staltz Yes! :D

Just for the sake of any future readers:

We ended up using the Comlink library to move the WASM initialization and WebWorker threadpool initialization off the main thread. We also introduced bundling with webpack (https://github.com/ssb-ngi-pointer/ssb-validate2-rsjs-wasm/pull/1) which conveniently offers us hassle-free Firefox support (no polyfill needed).