search

mymonero / mymonero-core-js

The JS library containing the Monero crypto plus lightwallet functions behind the official MyMonero apps
BSD 3-Clause "New" or "Revised" License
101 stars 103 forks source link

Added derivation_to_scalar #71

Closed gutenye closed 5 years ago

gutenye commented 5 years ago

Depends on https://github.com/mymonero/mymonero-core-cpp/pull/10/files Fixes https://github.com/mymonero/mymonero-core-js/issues/68

paulshapiro commented 5 years ago

Looks good! The only thing that's missing is rebuilding MyMoneroCoreCpp* (via bin/archive-emcpp.sh). I can handle that after merging this but I might wait a little (mentioned in other PR) as we have some incoming changes that will lead to a rebuild anyway. (Unless you need this rebuilt for your project – let me know)

gutenye commented 5 years ago

Thanks for the code review. I can wait for that, and for my project, I have rebuilt the MyMoneroCoreCpp* in my repo. I also added the decodeRctSimple bridge code, will send a seperate PR shortly.

paulshapiro commented 5 years ago

Hey @gutenye, could I trouble you to rebase? (Note we removed __bridged_fns_spec)

gutenye commented 5 years ago

@paulshapiro rebased

paulshapiro commented 5 years ago

So, @gutenye, I should have checked out your branch to test this - my bad - but you're actually missing the entire serial_bridge function for derivation_to_scalar. So this breaks compilation, as it stands. You're also missing the serial_bridge function for decodeRctSimple. Those would need to be added before index.cpp can find anything to reference.

By the way, are you able to compile the emscripten build locally so you can verify?

gutenye commented 5 years ago

Yes, I can compile the emscripten build locally. Since we're on the topic, I have a question about it. I found the sha256sum of the MyMoneroCoreCpp.wasm output is different on a fresh build each time (with same code but build multiples. Reproduce by: 1) clone a fresh repo 2) build it). Do you know what causes it?

paulshapiro commented 5 years ago

Hmm unfortunately I do not. I can have a look at docs and issues tmw to see if I can find anything.

paulshapiro commented 5 years ago

hey @gutenye! So I had a look into this. Even with my google-fu, I was unable to turn up terribly much, aside from this: https://github.com/WebAssembly/binaryen/issues/936 - There's a comment about nondeterminism in linking order, leading to different LLVM temp symbol names - but that should affect asm.js too. So I set about performing rebuilds and checking asm.js and wasm sha256sums to see if we would see variance in the MyMoneroCoreCpp_ASMJS.asm.js. And funny enough, I actually got the same shasum not only for the asm.js but for the wasm file as well! So it occurs to me - perhaps the non-determinism is within the boost sdk build phase? Because I was not rebuilding boost each time - only running archive-emcpp.sh. (For the record I was ensuring the build caches aside from boost were completely cleared so as to rule out actually shasumming the same files over and over).

gutenye commented 5 years ago

@paulshapiro Thanks for looking into it, I really appreciate. That's a good lead, I'll do more tests tomorrow and let you know the result.

paulshapiro commented 5 years ago

It's a pleasure. Fwiw I'm finding hints that boost might not build deterministically under some conditions. https://svn.boost.org/trac10/ticket/11111

paulshapiro commented 5 years ago

(And I could imagine that toolset=clang-emscripten might not have deterministic mode set by default)

gutenye commented 5 years ago

If boost is the cause, I think I can just build the boost for once and re-use it next time.

paulshapiro commented 5 years ago

Agreed. I'd love to solve this for all though.

gutenye commented 5 years ago

@paulshapiro I found the path difference cause boost not build deterministically.

  1. use /dira/mymonero-core-js, and run bin/build-boost-emscripten.sh, then sha256sum build/boost/lib/libboost_thread.a
  2. Move to /dirb/mymonero-core-js, rebuild with bin/build-boost-emscripten.sh, checksum is different.
  3. Move back to /dira/mymonero-core-js, rebuild boost, checksum is same as step1.

I just don't understand how a path changes it, do you have a solution?

paulshapiro commented 5 years ago

Hey I'm sorry that I didn't get back to you until just now – I didn't realize there was a question at the end of your message as I only saw the first part of it, thought you might dig into it a little, and then I forgot about replying.

I do recall something related to boost or possibly cmake deals with absolute paths. I'll try to have a look at this tomorrow and will let you know if I find anything.

gutenye commented 5 years ago

np, after the absolute path issue fixed, build is still not deterministic across the machine.

I tried to build with docker with difference machines in cloud services, following steps here: https://github.com/ExodusMovement/mymonero-core-js/blob/master/BUILD.md Boost build is deterministic, but the result is still different. Here's my result:

1. 2 Ubuntu 16.04 in Google Cloud build same output.
2. 2 CoreOS 1995 in Google Cloud build same output.
3. 1 Ubuntu 16.04 and 1 CoreOS 1995 in Google Cloud build different output.
4. 1 CoreOS 1995 in Google Cloud and 1 CoreOS 1995 Aliyun Cloud build different output.

The host running docker affects the result, I believe the problem is inside emscripten toolchain.

paulshapiro commented 5 years ago

Pretty strange though since my builds came out the same. Could emscripten be caching something at the system or user level?

paulshapiro commented 5 years ago

@gutenye here's a reply from the emscripten maintainer. Would you like to have a go at it ?

https://github.com/kripken/emscripten/issues/7714#issuecomment-449454785

gutenye commented 5 years ago

@paulshapiro Thanks for the help, will test it.