search

hugomrdias / rabin-wasm

Rabin fingerprinting implemented in WASM
28 stars 7 forks source link

Why was WASM impl chosen over pure JS #209

Open Gozala opened 2 years ago

Gozala commented 2 years ago

@hugomrdias @achingbrain I had been working on Rust (re)implementation of rabin chunker compatible with current go implementation https://github.com/Gozala/rabin-wasm/

Some shallow profiling shows suggests that most time is wasted copying bytes into WASM memory, which is perhaps unsurprising. In addition there is a downside of async initialization.

This got me wondering why WASM implementation was chosen over pure JS and if some perf comparison used in making that decision. Without any data to support this, I am inclined to think that pure JS implementation would likely perform better is it just needs to do shifts and xor operations and both seems to be supported on BigInt

achingbrain commented 2 years ago

From memory at the time we were trying to remove dependencies with native addons. We were using the rabin module from the DAT project which uses the C implementation from lbfs.

WASM was used because it was considerably less work than porting the algorithm to JS and also ensuring its correctness. Raw performance wasn't really a consideration.

It's not without it's issues though: for the same input, the output of this module doesn't always agree with the go-ipfs implementation, though I've never had the time to investigate which implementation is right and which is wrong.

It would be very useful to have these things in alignment, if you have the time to spend on it.