Bulletproofs

[dternyak]

On October 18th, Monero will hard-fork to support bulletproofs. Similar to the previous ringCT hardfork, transaction libraries like xmr-core will need to be updated to support the new transaction format introduced.

Countdown to hardfork for reference: http://xmr.noctism.com/

[ph4r05]

I could implement Bulletproofs, ut it would be great to have sc_inv() implemented meanwhile, i.e. scalar modular inversion (mod curve order).

[paulshapiro]

Why don't you use mymonero-core-js? We already transpile the bulletproofs code.

[ph4r05]

@paulshapiro I was looking for BP support in the repo but I only found https://github.com/mymonero/mymonero-core-js/blob/37c86c23e09f11742108e7f246fe8992082f18e0/cryptonote_utils/cryptonote_utils.js#L1894

I was wondering how emscriptem generated files were generated. Is there any info about it? Where is the source and command line to do it? I would either add sc_mul, sc_inv primitives to the emscriptem generated files and implement BP in the JS manually (as are Borromean range proofs right now) or transpile it as a whole. Thanks

[paulshapiro]

@ph4r05 that's JS from the old mymonero-core-js which was added by henrynguyen a couple months ago, just before we switched to C++, rather than the create tx function in the Monero C++ which then gets transpiled to wasm in the new mymonero-core-js. Check https://github.com/mymonero/monero-core-custom/blob/e089dec813e7c9b51b8ded08bf8017e0395c4155/cryptonote_core/cryptonote_tx_utils.cpp#L604

We then merely need to re-enable the boolean flag to turn it on or off from the client-side, or, probably better, we just update the fork rules to turn it on.

The source and command to do it? I haven't merged develop into master yet so it's on mymonero-core-js:develop. Here was the PR for who missed it. Emscripten works through cmake so check the CMakeLists.txt and bin/archive-emcpp.sh. You also need to build boost yourself, for which instructions exist in the mymonero-core-js develop branch readme. You don't need OpenSSL because we (i.e. Monero project) replaced the sc_inv with a manual implementation.

However the mainline bulletproofs code has received some minor changes so I will merge those soon.

All of this work is just to get building in your dev env, but the work that xmr-core would need in order to have bulletproofs support ledger is as I've described briefly in this reddit post: https://www.reddit.com/r/Monero/comments/9hqmm4/i_love_xmr_but_storing_it_is_a_btch/e6e0v3j/

i.e. even though Monero mainline has Ledger (hwdevice) support, as thus does this transpiled code, the transport etc code still exists in JS, so we need to hook into it from a new hwdevice.

I don't want to say it might be best for me to spearhead this but that may be most efficient. In any case we can freely collaborate on github or if you would like to drop into #mymonero on freenode IRC I think it would be beneficial for the other people who have been following developments as well.

[paulshapiro]

By the way here's the bridge for calling create transaction from JS/emscripten (or whichever other binding you would like) https://github.com/mymonero/mymonero-core-cpp/blob/master/src/serial_bridge_index.cpp#L501

You can ignore the comment about taking the tx sec key as an arg - that was prior to ensuring proper rand gen via an emscripten PR.

[paulshapiro]

All this stuff gets tested as well, e.g. https://github.com/mymonero/mymonero-core-cpp/blob/master/test/test_all.cpp#L323 , which you can run like this if you'd like https://github.com/mymonero/mymonero-core-cpp/blob/master/bin/buildAndRun_tests

That way we can ensure the C++ works first and simplify JS testing

[paulshapiro]

By the way on my todo list I've got re-enabling bulletproofs coming up very shortly anyway, so I think it makes sense for me to tackle it. But you've still got the Ledger work on top – something I don't plan to get to for a little while.

[ph4r05]

@paulshapiro wow, thanks for the valuable information! I will go through it and get back later.

So I leave Bulletproofs and focus on my primary interest now - Trezor integration.

[dternyak]

@ph4r05 Cool - Trezor integration would be sweet too, especially since they seem to be close to release.

Are you planning on integrating Trezor into mymonero-core then? If so, it would be worthwhile to hit Ledger as well, since their device is already working with Monero today.

I still think there are some advantages to porting bulletproofs to xmr-core, since you get the benefits of type safety via typescript, as well as a working ledger implementation already.

Really cool to see more contributors on this!

[ph4r05]

@dternyak regarding the Bulletproofs after giving it a second thought I think it is better to have it implemented in web assembly as it will be significantly faster rather than having it implemented in JS. JS implementation could be nice for educational purposes but for practical use wasm implementation is better, I think.

There can still be a TS thin wrapper for the safety.

[p0o]

I believe this repo should be kept going since it contains a better-organized codebase and has the main features implemented in JavaScript. Having implementations in JS will open up Monero to more JS developers rather than asking them to work with a C++ transpiled codebase. The wasm approach of MyMonero is great for short term productivity but not good for community growth IMHO

[ph4r05]

I like this repo and the JS Monero codebase too. But it needs someone taking care of this for a long time. Because of this I personally think the wasm approach is more sustainable as Monero C++ codebase is in a long-term active development and maintenance costs for wasm port are lower.

Anyway, I have already ported Bulletproofs to Python in https://github.com/ph4r05/monero-agent/blob/master/monero_glue/xmr/bulletproof.py

If somebody manages to make sc_mul, sc_muladd available I could at some point port Bulletproofs to JS as well. It is useful at least for educational purposes.

[dternyak]

@ph4r05 I think it's also useful for the immediate availability of Ledger web wallet support, but I do see your point about piggybacking on the development of the C++ codebase.

[p0o]

@ph4r05 Current crypto ops at the moment are not implemented in pure JavaScript and are being called from this emscripten file. I was thinking maybe with having a well tested int64_t and uint64_t data types implemented here we can re-implement most of the crypto ops functions including sc_mul and sc_mul_add in pure JS. I can help with the heavy lifting of doing same operations in JS but I need help with testing since my foundations are weak and don't really understand all these numbers in the original monero implementation 😩

[ph4r05]

@p0o the crypto operations could be also implemented using just 32bit numbers but I think this pure JS crypto backend would make implementation unusable due to performance penalty.

I like the emscriptem / wasm approach with the low-level crypto stuff. I don't know how was the file generated and how to re-generate it so it includes Bulletproof needed operations.

[paulshapiro]

@ph4r05 don't worry, i'll be doing that shortly

[paulshapiro]

the file generation instructions are all in the readme in any case.. there are scripts etc... see mymonero core cpp and mymonero core js repos

[p0o]

@ph4r05 do you have any benchmark or estimation that how bad the JS implementation could be? AFAIK the emscripten approach with compressed/uglified code can have CSP implications for more restrict environments like chrome extensions.

[paulshapiro]

@p0o A few things to clarify..

1. I dont know running wasm in chrome extensions myself but the CSP requirements for loading wasm (unsafe eval) are something that Chrome, Mozilla, et al have been working on (there are a number of articles and github threads about it) and which the entire ecosystem is subject to. There are plans for how to address it, e.g. unsafe-wasm rather than full eval.

2. The emscripten approach is not to "compress/uglify". The emscripten JS module code just some basic JS for wrapping devices etc. I would recommend reading up a bit on how emscripten and binaryen work as they're not compressing/uglifying but compiling C/C++/Rust etc using an llvm front end which the browser directly executes. No one reads compiled wasm.

3. Benchmarking the JS implementation sort of misses the point. Wasm is bound to be faster with really optimized code anyway. You would have to handwrite bulletproofs in JS in such a way not only to support all the multiexp etc other modifications that are being done to it, but you would have to fine-tune it to meet similar optimizations done in the Monero src, llvm, and browsers.

[ph4r05]

@paulshapiro just one thought on point 3 - you don't actually need Straus / Pippenger multiexp optimizations on the user wallet side. On Trezor we dropped it as it does not bring that significant speedup considering the RAM overhead. User wallet does not need to perform blockchain bulletproof verification so the implementation similar to what I did in Python would work.

[paulshapiro]

Just for the record I myself don't actually think bulletproofs construction is really that performance sensitive here anyway. It could take 300ms and the user would barely notice.

[paulshapiro]

@ph4r05 gotcha

[p0o]

@paulshapiro my bad for using the word uglified but based on a recent blog post from Google as far as I understood a file like this would face auditing problems for chrome web store. Also, the trust factor plays a role here since no one really knows what's happening in that kind of code. WASM tho should be fine.

I get the point about challenges for JS implementations but as a person with more app development experience than Crypto experience, I think we can't see Monero adaption through apps if we don't have a fully working implementation with JS. Developers need to read the code to know what they can do with it.

[paulshapiro]

the trust factor plays a role here since no one really knows what's happening in that kind of code. WASM tho should be fine

There's the potential for reproducible builds and signing, and the browser runtime literally steps through the instructions... that's basically like saying that we don't know what's happening in compiled code... hm, did you read how emscripten works?

as a person with more app development experience than Crypto experience

Same boat!

I think we can't see Monero adaption through apps if we don't have a fully working implementation with JS. Developers need to read the code to know what they can do with it.

I don't see how that follows or how the premise that devs can't see the code is accurate. All the code is laid out in JS and C++. The bridge code is extremely simple and quite sufficiently performant. And we also have source mapping analogues on wasm. I myself have obviously made considerable inroads into it (and might I add, development didn't take place through the emscripten build process? there are boost tests for these things), so the conclusion that devs can't know what they can do with it frankly doesn't seem to follow, to me.

Besides, it's not obvious that it's actually sustainable for app developers to have to reimplement their own cryptographic schemes in isolation rather than attempting to use standards and existing battle tested implementations (a big no-no in security software). A quick survey of Monero app devs will quickly turn that up. This not simply a matter of whether to implement a singular new scheme (bulletproofs). It's a matter of whether to build all of the crypto yourself and who is going to actually keep up with the updates and audit it.

Anyway, I don't see a reason why I should have to discourage anyone from implementing bulletproofs in their language of choice, so please feel free, it's not like I could ask you to spend your time on hooking up existing hw device support in mymonero-core-js instead! Let me know if you find out someone ends up writing it, hey? I'd be interested in looking at it.

By the way @p0o what plugin are you working on?