Open nazar-pc opened 5 years ago
vkomenda
commented
5 years ago Hi @nazar-pc. What kind of difficulties do you have with glibc and gmp? We can help. It will be great to keep the work in the upstream repo, so please do send your PR when you think it's ready.
nazar-pc
commented
5 years ago With glibc the issue is that it can't be compiled to wasm, it outputs something like this:
From what I found online, musl is a better option when compilation to WASM is needed, since there is simply no implementation for these for wasm32 target.
Things are a bit better with cargo build --target=wasm32-unknown-emscripten, but it still complains about being unable to find gmp library and a bunch of related symbols:
I ran it in debug mode and crafted compilation command for Emscripten manually by specifying path to custom build of libgmp.a and it seems to compile now (yet, have to test if it does anything). Not to mention, all of this happens in specially crafted Docker container on top of i386 image of Ubuntu 18.04.
In general, it seems to be much easier to deal with pure Rust code rather than with C libraries as dependencies.
I should add that I'm not a Rust developer and I can barely read it for now 🙂
nazar-pc
commented
5 years ago So, it works, but performance is (3 iterations)...
root@2822aa35515b:/src# time node target/wasm32-unknown-emscripten/release/deps/vdf-e19f32839fa6ad1a.js
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe5000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001
real 1m8.782s
user 1m8.936s
sys 0m0.024sSo, it works, but performance is (3 iterations)...
Great. Can you try in release mode? Because GMP isn't the fastest option anyway (#13), we'll need a replacement at some point. Of course it would be great to have a numeric library in Rust but for now it is external.
nazar-pc
commented
5 years ago It is release mode. With compiler optimization options it is already down to ~12 seconds, but I'm not sure it would be possible to go much further after that very easily.
vkomenda
commented
5 years ago OK. We need to try to build a static musl binary then.
nazar-pc
commented
5 years ago I figured out a relatively straightforward way to compile it, one of the last things is that I have difficulties with cross-language interaction. Any help is very much appreciated: https://github.com/emscripten-core/emscripten/issues/8441
afck
commented
5 years ago Awesome! Not sure whether I can be of much help, but I'm really curious what the slowdown will be, if you compile GMP and vdf to WebAssembly, compared to native code. (I've read encouraging numbers about WebAssembly performance, but still: If it's 5 times slower, that makes a WebAssembly vdf 5 times less useful, I guess?)
nazar-pc
commented
5 years ago VDF is surprisingly slow right now, I'll finish tests and docs and will publish to separate repo first. My guess is that it might be related to math with large numbers or something like that (remember, wasm is 32-bit only right now). But once we have something that works, we can hopefully profile it and find out why exactly it is so slow and make it faster.
afck
commented
5 years ago To me it sounds like, if that's feasible in your use case, it would be better to somehow expose the native VDF functionality as a foreign function to WebAssembly. Otherwise the speed advantage of a hypothetical attacker has to be multiplied by the WebAssembly slowdown.
nazar-pc
commented
5 years ago @afck WebAssembly is a bit different. It is just like it sounds - assembly for the web. So FFI in traditional sense is possible in Node.js with native bindings and we at Subspace will look at this direction too. But in any case we need this thing to run in modern browser (just like most other components necessary for the protocol), which is why I pursued with compilation to WebAssembly first.
Since there is some interest, here is undocumented (currently) repo that actually works: https://github.com/subspace/vdf/blob/master/src/main.rs
You can run ts-node test.ts (you need to have ts-node installed on your machine) to test basic use case.
If anyone has skills to debug WebAssembly, I would be happy to work on that together, otherwise I'll try to do something myself.
afck
commented
5 years ago Right, if it needs to run in the browser, there's no way around compiling everything to WebAssembly, of course. :+1:
Hi, I'm trying to compile VDF library to WebAssembly so that it can be used in Browser and Node.js enviroments for Subspace project. I'm having a lot of difficulties with this endeavor (some of the major pain points being glibc and gmp libraries), but hoping to get it working eventually.
Would you be interested in having WebAssembly support in this repo, helping in any way or any other thoughts on the topic? I'd prefer to upstream all the work in this direction so that it can be kept up to date with any changes happening to source code.