Add WASM compilation support for verkle prover

[acolytec3]

The verkle-trie module currently cannot be compiled to WASM because it depends on the rocksdb bindings which are from C (if I'm reading the logs correctly). I'm not sure if there's any direct way around this unless there is a pure Rust version of rocksdb that could be compiled to WASM as well. Would it be possible to separate out just the verkle verifier code into a standalone module that isn't dependent on rocksdb so it could be used in the browser?

By adding the below to dependencies

[dependencies]
...
wasm-bindgen = "0.2"
getrandom = { version = "0.2", features = ["js"] }

and then

[lib]
crate-type = ["cdylib", "rlib"]

I was able to get to the point of the rocksdb failure that generates the below error by running wasm-pack build --target web in the root of the verkle-trie package.

error: failed to run custom build command for `librocksdb-sys v6.20.3`

Caused by:
  process didn't exit successfully: `/home/jim/development/rust-verkle/target/release/build/librocksdb-sys-80b89031e3c367eb/build-script-build` (exit status: 101)
  --- stderr
  rocksdb/include/rocksdb/c.h:65:10: fatal error: 'stdarg.h' file not found
  rocksdb/include/rocksdb/c.h:65:10: fatal error: 'stdarg.h' file not found, err: true
  thread 'main' panicked at 'unable to generate rocksdb bindings: ()', /home/jim/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-6.20.3/build.rs:44:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[ryanio]

thanks for giving this an initial try @acolytec3! agree with your reasoning that if the state access could be extracted with a put/get API, we could supply our own db and reduce the need for the rocksdb dependency (we use leveldb in ethereumjs in both Node and the browser)

edit: oh! rocksdb is an abstract-leveldown compliant store by level. i didn't know that! in that case could we simply pass you a leveldown compliant store as an arg? this is already how our trie, vm, and client packages handle their dbs so that would be simple.

[acolytec3]

https://github.com/crate-crypto/rust-verkle/blob/master/verkle-trie/src/database.rs If I'm reading the comments here, what would be needed is just an alternative implementation of the two methods exposed here declared as extern and then we create methods on the JS side that provide the needed DB methods, right?

[kevaundray]

Hi, thanks for raising this issue. rust-verkle does not need rocksdb or verkle-db, so it should be put as optional.

rust-verkle only requires that you implement the methods in the file that you linked above. If you have a key-value database, then you can implement the methods in verkle-db and those methods in in database.rs will use default implementations.

[kevaundray]

thanks for giving this an initial try @acolytec3! agree with your reasoning that if the state access could be extracted with a put/get API, we could supply our own db and reduce the need for the rocksdb dependency (we use leveldb in ethereumjs in both Node and the browser)

edit: oh! rocksdb is an abstract-leveldown compliant store by level. i didn't know that! in that case could we simply pass you a leveldown compliant store as an arg? this is already how our trie, vm, and client packages handle their dbs so that would be simple.

Yep!, although instead of a put/get API, we have a "batch_put" and a get api, because we only ever write to the disk in batches/transactions.

[kevaundray]

This should be fixed now on main, I tested the wasm-pack command you mentioned above and it produces a .wasm file along with accompanying typescript and javascript files.

[acolytec3]

Just to verify what I need, I just need to implement externs for the methods in this file, right?

[kevaundray]

Sorry I missed the last message.

I'm not familiar enough with WASM to comment on that part with 100% confidence, but I believe you are correct, because if you were using this library natively in Rust as a dependency, you would only need to implement those methods.

To test this hypothesis without the complexity of this codebase, you could have a simpler rust project with only the following trait:

pub trait Database {
    fn put(key : [u8;32], value : [u8;32]);
    fn get(key : [u8;32]) -> Option<[u8;32]>;
}

And provide bindings through WASM for these methods initially

[acolytec3]

Thanks! I'm a complete novice with Rust but can follow "how tos" okay so we'll see how far I can get. I can confirm I was able to compile the current master branch for WASM which is promising.

For anyone else trying to compile verkle-trie for WASM, please note the following requirements to compile:

1) in Cargo.toml, add the following two pieces.

[Dependencies]
...
[dependencies]
...
wasm-bindgen = "0.2"
getrandom = { version = "0.2", features = ["js"] }
...

[lib]
crate-type = ["cdylib", "rlib"]

2) Ensure you run rustup default nightly

3) Run wasm-pack build --target web from [repo_root]/verkle-trie to compile the module.

[kevaundray]

No problem :) let me know if anything else looks weird!

[acolytec3]

FYI - I've started a fork to work on a WASM compatible implementation attempting to implement a jsdb_impl here.

It doesn't compile yet but I think that's at least as much ascribable to my ineptitude with Rust as anything.

[holgerd77]

Hi @kevaundray, thanks for giving this a start! 🙂

[kevaundray]

Here is the relevant repo which compiles the Rust code into Wasm and has an example js file which calls the generated wasm with js generated values: https://github.com/crate-crypto/rust-verkle-wasm

The root that is being returned is a stub until actual values can be taken received from the Go code

[holgerd77]

Hi @kevaundray, that's really great, thanks a lot! 🎉 😀