Open paulmillr opened 5 months ago
We are currently looking into replacing our rustbn-wasm build with this library for the bn254
(bn128) ADD, MUL, PAIRING precompiles from EIP-196 and EIP-197, so I agree that adding support for "bn254 g2, bn254 pairing" would be great idea! (respectively pretty valuable to us)
(side question: do you think we could use Noble for the BLS precompiles (so https://eips.ethereum.org/EIPS/eip-2537 ?)
Yes, bls precompiles are ok.
bn254 will need to be added
@holgerd77 the performance would be decreased compared to wasm. For example, bls12 does 100 pairings/sec on fast mac. For comparison, ETH can do 4000 signatures/sec on the same cpu. Your current rustbn wasm solution is likely several times faster.
How constrained are you in terms of performance?
Cool!
For BLS I've compiled down the following list of potential function mappings, if you have something to add:
mcl.add -> bls12_381.fields.Fp.add
mcl.mul -> bls12_381.fields.Fp.mul
mcl.mulVec -> ? (bls12_381.fields.Fp.mulN ?)
mcl.millerLoop -> bls12_381.millerLoop
mcl.finalExp -> ?
mcl.G1/G2 -> bls12_381.G1/G2
mcl.Fp(2) -> bls12_381.fields.Fp (2)
mcl.verifyOrderG1/G2 -> bls12_381.G1.isWithinCurveOrder
We will go our "classic" route here, and provide the JS implementation as default and then let people opt-in (dependency-inect) WASM if they want to.
Actually with the pairing and if we could replace bn254 we would be completely "WASM free" for the EVM which would be a pretty big deal! 🤩
I will make it a priority then.
Great to hear (if you need financial support for this let me know)! 🙏 🤩
bn254 pairings have been added.
Ok, I am on it, see above PR!
Have now created a new build here https://github.com/holgerd77/noble-curves/tree/build-3ed792f . When building our EVM code I am getting two BLS related errors now, not sure if something to fix on your or on our side, will look into it:
Update: think this is just because of conflicting noble versions (now having noble from my branch + noble from ethereum-cryptography in). Can solve this with any
annotation and remove later.
toHex()
on the point seems not implemented (throws with "not implemeneted")
Same with toRawBytes()
Ok, first Noble usage for multiplication seems to be (at least partially) working https://github.com/ethereumjs/ethereumjs-monorepo/pull/3564/files#diff-d8e4e56024eab6bc6446b026499696dcfcc4cbb9b6bdfe6c9fd35e983052d026 🙂.
Replaced the rustbn.js WASM code and this locally passes some ec_mul
tests (ecmul_7827-6598_9_21000_128
) from the official ethereum/tests
test suite.
seems not implemented
Reason: there is no standard, everyone does different things. I will look at what you're doing here and maybe make tohex the same.
In between state: I have now got multiplication and addition working. This should be tested enough, there are 100s (~1000 or so?) tests in the official ethereum/tests
which all pass now. Will now move over to pairing.
Hi Paul, I am currently trying to create a valid G2 point, which is not working yet. Can you have a look at the following code, guess you will spot quickly what is wrong?
import { hexToBytes } from '@ethereumjs/util'
import { bn254 } from '@noble/curves/bn254'
// From ecpairing_two_point_match_1 test
/* const input =
'0x00000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000002198e9393920d483a7260bfb731fb5d25f1aa493335a9e71297e485b7aef312c21800deef121f1e76426a00665e5c4479674322d4f75edadd46debd5cd992f6ed090689d0585ff075ec9e99ad690c3395bc4b313370b38ef355acdadcd122975b12c85ea5db8c6deb4aab71808dcb408fe3d1e7690c43d37b4ce6cc0166fa7daa000000000000000000000000000000000000000000000000000000000000000130644e72e131a029b85045b68181585d97816a916871ca8d3c208c16d87cfd45198e9393920d483a7260bfb731fb5d25f1aa493335a9e71297e485b7aef312c21800deef121f1e76426a00665e5c4479674322d4f75edadd46debd5cd992f6ed090689d0585ff075ec9e99ad690c3395bc4b313370b38ef355acdadcd122975b12c85ea5db8c6deb4aab71808dcb408fe3d1e7690c43d37b4ce6cc0166fa7daa' */
console.log('test')
const inputG2 =
'0x198e9393920d483a7260bfb731fb5d25f1aa493335a9e71297e485b7aef312c21800deef121f1e76426a00665e5c4479674322d4f75edadd46debd5cd992f6ed090689d0585ff075ec9e99ad690c3395bc4b313370b38ef355acdadcd122975b12c85ea5db8c6deb4aab71808dcb408fe3d1e7690c43d37b4ce6cc0166fa7daa'
const inputG2Bytes = hexToBytes(inputG2)
console.log(inputG2Bytes.length)
const pFP1 = bn254.fields.Fp2.fromBytes(inputG2Bytes.slice(0, 64))
const pFP2 = bn254.fields.Fp2.fromBytes(inputG2Bytes.slice(64, 128))
const pG2 = bn254.G2.ProjectivePoint.fromAffine({
x: pFP1,
y: pFP2,
})
pG2.assertValidity()
So this is the test input from one of the tests, input
is the full input passed to the precompile and inputG2
the part I am using for G2 (obviously contained in input
).
I am trying to get a positive assertValidity()
check. Atm it is always throwing there.
Ok, I think I've found the respective code parts in your tests, will try to take it from there.
Ok, all official tests passing (I basically did the same as you did in your EVM setup just not in elegant 😂). https://github.com/ethereumjs/ethereumjs-monorepo/pull/3564/
Think this should be enough for a release, at least from my side! So the test suite from ethereum/tests
should really be complete, since the EIPs are out for years.
The only thing which irritated me mildy is that I had to flip the elements for the G2 point creation, see: https://github.com/ethereumjs/ethereumjs-monorepo/pull/3564/files#diff-d8e4e56024eab6bc6446b026499696dcfcc4cbb9b6bdfe6c9fd35e983052d026R83
Not sure if this is an inconsistency in your library or in the EIP or otherwise has some reason I do not understand. If this is something to be adjusted on your side you can simply do along release I guess without another testing round.
Again, thanks a lot for all the work on this as well as this thorough EVM-specific test setup! 🙏 🎉
It's the same issue as in with toHex: there is no standard for this. Someone can serialize imaginary part of the complex number (G2) first, someone does the opposite.
For comparison, BLS was specced with hash-to-curve, so it's much better.
I will see what can be improved for your cases and prepare new release today/tomorrow.
There would be no serialization by default for now. Reasoning:
EVM picked: BE, no-flag, imag-first. Which is not compatible with something like ZEC and less feature-full. For example, no one else does BE.
I will add it to the readme. The community can then figure out the optional path of moving forward.
1.5.0 is out.
Do you need a new release of eth-cryptography today? 1.5.0 can be used as-is, but will be duplicated in your dep tree for a few weeks.
After the audit, I will include bn254 into eth-cryptography (all packages there are audited).
Cool, will test! 🤩
Not including yet into eth-cryptography might counterbalance some of our tree shaking efforts for the moment (not fully sure), since we then might get double code parts from the different versions in.
Guess nevertheless makes sense (not to include yet). So, as you said, audit is not so far away, right?
Audit is this month. So even if you do major releases today, you can do x.y.1 patch to improve tree-shaking later this month.
Ah, that’s great, no no, we are still a month away from releases
In preparation for incorporation into ethereum-cryptograhy
: https://github.com/paulmillr/scure-bip32/pull/17
@paulmillr Do you think this library will be able to be compiled with Static Hermes?
I’m not sure what are the constraints. AFAIK, many people use it with react native.
I’m not sure what are the constraints
Sound types
AFAIK, many people use it with react native.
Static Hermes isn't released yet. Static Hermes compiles JS to native instructions ahead of time.
Here is some context:
https://tmikov.blogspot.com/2023/09/how-to-speed-up-micro-benchmark-300x.html https://speakerdeck.com/tmikov2023/static-hermes-react-native-eu-2023-announcement https://x.com/tmikov/status/1700353858763911570?s=20
Basically noble will be THE fastest cryptography library on React Native without native code
What's "sound types" and how does one detect which types are unsound? Is there automatic tool?
@paulmillr
What's "sound types" and how does one detect which types are unsound? Is there automatic tool?
Essentially if you have type number
annotation on a variable in JS runtime it can become undefined
or string
or whatever. Instead of silently failing or basically continuing to work Static Hermes will throw just like in any strongly typed language. So if the noble
depends on JS looseness it must be refactored.
I'm open to a speed up, but there should be some simple tool to check all files for "sound type" errors.
@paulmillr that's a good point. I assume there will be something like that. I'm gonna try to find out
@paulmillr Ok I went ahead and tried to compile https://github.com/paulmillr/noble-secp256k1/blob/main/index.js . I chose secp256k1
because I'm already familiar with the lib.
mkdir hermes_build
cd hermes_build
git clone https://github.com/facebook/hermes
_Taken from: https://github.com/facebook/hermes/blob/static_h/doc/BuildingAndRunning.md_
cmake -S hermes -B build -G Ninja
cmake --build ./build
To build release:
cmake -S hermes -B build_release -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build ./build_release
index.js
with Hermesmodule.exports
) the last 2 export lines since Hermes doesn't support ES6 for nowbuild/bin/shermes --typed -exec ../noble-secp256k1/index.js
The --typed
instruction will try to compile the file. That's essentially how you verify if you have sound or unsound types. More info about that
@paulmillr have you had a chance to check static hermes out?
@alexandrius the build commands do not produce any shermes
file and with hermes
--static
flag does not work
@paulmillr did you checkout to static_h
branch? I missed the part in the command
I have removed all initial errors (see https://gist.github.com/paulmillr/d9d5fdf8d8c9d7c954e9f26fdd30bc9d) but then compiler emitted 20 more errors.
Seems like it's unreasonably complex for now.
There are no docs or guidelines or example how to fix stuff. For example I have property s not defined in class Signature
- but it's clearly specified in constructor. ETc.
I've wrote a bunch of formulas to fit everything in 100 lines. Even this code doesn't work and is constantly segfaulting. https://gist.github.com/paulmillr/8002c140bec8ea64fd3b5efcf867eaf1