search

autonomys / vdf

An implementation of Verifiable Delay Functions compiled from Rust to WebAssembly
MIT License
9 stars 2 forks source link

How to increase the speed? #1

Open jio-gl opened 3 years ago

jio-gl commented 3 years ago

Is true is one order of magnitude slower than the Rust. Any ideas on how to match the Rust impl?

Thanks, J.

nazar-pc commented 3 years ago

It was according to my tests.

I don't think you'll be able to match Rust. You may try newer Rust compiler, the one that was used here is pretty old and I was very new to Rust back then. There is also SIMD in WASM at least in some browsers, so that may or may not be useful to try as well.

jio-gl commented 3 years ago

thanks.

jio-gl commented 3 years ago

I have an issue compiling with GMP. I am trying Ubuntu, I can try in MacOS too...

libgmp.a is not built yet, compiling it
./build.sh: line 29: pushd: too many arguments

Here:

   pushd $CACHE_DIR
        curl https://gmplib.org/download/gmp/$GMP_RELEASE.tar.bz2 -O
        tar xf $GMP_RELEASE.tar.bz2
        pushd $GMP_RELEASE
            CC_FOR_BUILD=$(which gcc) emconfigure ./configure --build i386-linux-gnu --host none --disable-assembly --disable-shared --prefix=$(pwd)/build
            patch < ../../src/gmp/config.h.patch
            make -j $(getconf _NPROCESSORS_ONLN)
            cp .libs/libgmp.a $CACHE_DIR/lib/
        popd
        rm -rf $GMP_RELEASE*
    popd

Thanks.

nazar-pc commented 3 years ago

I think the path to your project folder has whitespaces or special characters, dumb script wasn't designed to handle them :upside_down_face: Try compiling from within directory that doesn't have them.

jio-gl commented 3 years ago

I am compiling on Ubuntu now, will see... Thanks a lot.

jio-gl commented 3 years ago

Could not link with GMP on WASM:

          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> void in /media/psf/Home/lab/Soliton/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.4.rcgu.o)
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> i32 in .cache/lib/libgmp.a(export.o)

          wasm-ld: error: function signature mismatch: __gmpz_tdiv_q_ui
          >>> defined as (i32, i32, i32) -> void in /media/psf/Home/lab/Soliton/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_q_ui.o)

          wasm-ld: error: function signature mismatch: __gmpz_tdiv_r_ui
          >>> defined as (i32, i32, i32) -> void in /media/psf/Home/lab/Soliton/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_r_ui.o)
nazar-pc commented 3 years ago

Try updating version in https://github.com/subspace/vdf/blob/master/build.sh#L22 to 6.2.1, which seems to be the latest release, probably vdf crate updated its dependencies as well (and don't forget to remove .cache directory so it builds gmp again.

jio-gl commented 3 years ago

There is something with the hardlinks:

copying selected object files to avoid basename conflicts...
libtool: link: ln mpz/abs.o .libs/libgmp.lax/lt1-abs.o || cp mpz/abs.o .libs/libgmp.lax/lt1-abs.o
ln: failed to create hard link '.libs/libgmp.lax/lt1-abs.o' => 'mpz/abs.o': Operation not permitted
libtool: link: ln mpz/add.o .libs/libgmp.lax/lt2-add.o || cp mpz/add.o .libs/libgmp.lax/lt2-add.o
ln: failed to create hard link '.libs/libgmp.lax/lt2-add.o' => 'mpz/add.o': Operation not permitted
libtool: link: ln mpz/add_ui.o .libs/libgmp.lax/lt3-add_ui.o || cp mpz/add_ui.o .libs/libgmp.lax/lt3-add_ui.o
ln: failed to create hard link '.libs/libgmp.lax/lt3-add_ui.o' => 'mpz/add_ui.o': Operation not permitted
libtool: link: ln mpz/clear.o .libs/libgmp.lax/lt4-clear.o || cp mpz/clear.o .libs/libgmp.lax/lt4-clear.o
ln: failed to create hard link '.libs/libgmp.lax/lt4-clear.o' => 'mpz/clear.o': Operation not permitted
libtool: link: ln mpz/clears.o .libs/libgmp.lax/lt5-clears.o || cp mpz/clears.o .libs/libgmp.lax/lt5-clears.o
ln: failed to create hard link '.libs/libgmp.lax/lt5-clears.o' => 'mpz/clears.o': Operation not permitted
libtool: link: ln mpz/cmp.o .libs/libgmp.lax/lt6-cmp.o || cp mpz/cmp.o .libs/libgmp.lax/lt6-cmp.o
...
jio-gl commented 3 years ago

Tried with GMP 6.2.1

nazar-pc commented 3 years ago

Not sure what it is, you'll have to do some experiments then

jio-gl commented 3 years ago

I think is a VM issue, the Ubuntu is a VM. Will try again in a not shared folder.

jio-gl commented 3 years ago

The WASM linker is too strict, maybe I need to use an older version of the emsdk ?

wasm-ld: error: function signature mismatch: __gmpz_export
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> void in /home/parallels/Desktop/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.4.rcgu.o)
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> i32 in .cache/lib/libgmp.a(export.o)

          wasm-ld: error: function signature mismatch: __gmpz_tdiv_q_ui
          >>> defined as (i32, i32, i32) -> void in /home/parallels/Desktop/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_q_ui.o)

          wasm-ld: error: function signature mismatch: __gmpz_tdiv_r_ui
          >>> defined as (i32, i32, i32) -> void in /home/parallels/Desktop/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_r_ui.o)
jio-gl commented 3 years ago

Tried disabling a flag due to 3 function mismatch, but I cannot compile subspace-vdf to WASM. Any clues? 

           CC_FOR_BUILD=$(which gcc) emconfigure ./configure --build i386-linux-gnu --host none --disable-assembly --disable-shared --prefix=$(pwd)/build CFLAGS='-s ERROR_ON_UNDEFINED_SYM\
BOLS=0' CXXFLAGS='-s ERROR_ON_UNDEFINED_SYMBOLS=0'
            patch < ../../src/gmp/config.h.patch
            make -j $(getconf _NPROCESSORS_ONLN) CFLAGS='-s ERROR_ON_UNDEFINED_SYMBOLS=0' CXXFLAGS='-s ERROR_ON_UNDEFINED_SYMBOLS=0'

  = note: emcc:WARNING: --llvm-lto ignored when using llvm backend
          system_libs:WARNING: main() is in the input files, but "_main" is not in EXPORTED_FUNCTIONS, which means it may be eliminated as dead code. Export it if you want main() to run.
          wasm-ld: error: function signature mismatch: __gmpz_export
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> void in /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libclassgroup-d76e008ce2b14c87.rlib(classgroup-d76e008ce2b14c87.classgroup.amxomvt1-cgu.4.rcgu.o)
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> i32 in .cache/lib/libgmp.a(export.o)

          wasm-ld: error: function signature mismatch: __gmpz_tdiv_q_ui
          >>> defined as (i32, i32, i32) -> void in /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libclassgroup-d76e008ce2b14c87.rlib(classgroup-d76e008ce2b14c87.classgroup.amxomvt1-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_q_ui.o)

          wasm-ld: error: function signature mismatch: __gmpz_tdiv_r_ui
          >>> defined as (i32, i32, i32) -> void in /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libclassgroup-d76e008ce2b14c87.rlib(classgroup-d76e008ce2b14c87.classgroup.amxomvt1-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_r_ui.o)
          emcc: error: '/home/parallels/lab/VDF/emsdk/upstream/bin/wasm-ld -o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.wasm -L/home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib -L/home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/self-contained /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.0.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.1.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.2.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.3.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.4.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.5.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.6.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.7.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.35gafff193gzq50j.rcgu.o -L/home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps -L/home/parallels/lab/VDF/vdf/target/release/deps -L/home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libvdf-48efadb155ccd181.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libsha2-d8f34aa3ce58a162.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libdigest-05bbe2952105514e.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libopaque_debug-0e4ecea6fa489716.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libfake_simd-8f252f8b25b74a91.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libblock_buffer-49d734868c895388.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libgeneric_array-20013f52121094ec.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libtypenum-11f9fa3a864b9b2f.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libblock_padding-b45aec8ca2ef2d2b.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libbyte_tools-e7622d9515268345.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libbyteorder-2fc6ade1e9b79472.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libclassgroup-d76e008ce2b14c87.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libnum_traits-2165ba5925761581.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/liblibc-949c035481ff33b9.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-2d3ec1ab9910bb82.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libpanic_unwind-910e7560fa396a1d.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_demangle-f173843d247e78a2.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libhashbrown-b05e05efe60b4f19.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_alloc-f6ae4ab5a9bc2d08.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libunwind-9f8b5f6efeb94207.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcfg_if-8c4a64bedea9d0fc.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liblibc-27afec8da4d3cd01.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liballoc-2f94ab43b6c4843a.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_core-018092b98e265aa8.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcore-49726ec2e30e61f2.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcompiler_builtins-ccd75a5f965f90ed.rlib .cache/lib/libgmp.a .cache/lib/libgmp.a .cache/lib/libgmp.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc.a -L.cache/lib --fatal-warnings -L/home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libgl.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libal.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libhtml5.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libcompiler_rt.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc++.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc++abi.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libdlmalloc.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc_rt_wasm-optz.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libsockets.a -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --allow-undefined --strip-debug --export malloc --export free --export generate --export verify --export emscripten_stack_get_end --export emscripten_stack_get_free --export emscripten_stack_init --export stackSave --export stackRestore --export stackAlloc --export __wasm_call_ctors --export fflush --export __errno_location --export __cxa_is_pointer_type --export __cxa_can_catch --export ntohs --export htonl --export htons --export emscripten_main_thread_process_queued_calls --export sysconf --export setThrew --export _get_tzname --export _get_daylight --export _get_timezone --export-table --export __start_em_asm --export __stop_em_asm -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024' failed (1)
nazar-pc commented 3 years ago

Either GMP or something else has changed, but I have no time to dig into it and figure out why unfortunately.

jio-gl commented 3 years ago

Will keep you posted.