search

ethereum / go-ethereum

Go implementation of the Ethereum protocol
https://geth.ethereum.org
GNU Lesser General Public License v3.0
47.46k stars 20.09k forks source link

EIP-4844 progress #27047

Closed karalabe closed 2 months ago

karalabe commented 1 year ago

This is a tracking issue for EIP-4844. Some of the issues might get broken down into very low level piecemeal ones that might help review and/or work on a different larger ones. Ideally we'll end up with a batch of PRs people can take a look at to wrap their head around 4844 and it's progress / coding history.

Note, an initial proof of concept was prepared by Coinbase and Optimism at https://github.com/ethereum/go-ethereum/pull/26283.

jtraglia commented 1 year ago

For KZG implementations, there are two options for Go clients:

Right now, our recommendation is for Go clients to use Go-KZG-4844. While C-KZG-4844 is expected to be slightly faster when run on single-threaded systems, Go-KZG-4844 also provides parallelized functions that can be faster (though less efficient) on systems with multiple threads. Most clients will use C-KZG-4844 bindings, but that's because there isn't a great alternative for them (right now). From a diversity perspective, using a different implementation would be beneficial. Both implementations will be audited and have been differentially fuzzed against each other.

Also, some information about the audit. There will be a one month audit (starting on April 17th 2023) for both of these implementations (and their bindings) by a reputable security firm. For Go-KZG-4844, code from gnark-crypto that it needs but was not audited has been copied into its project so it gets audited.

Benchmarks for Go-KZG-4844 With parallelization enabled on a system with 20 cores: ``` Benchmark/BlobToKZGCommitment-20 307 3901151 ns/op Benchmark/ComputeKZGProof-20 254 4697532 ns/op Benchmark/ComputeBlobKZGProof-20 246 4861995 ns/op Benchmark/VerifyKZGProof-20 858 1390373 ns/op Benchmark/VerifyBlobKZGProof-20 585 2032785 ns/op Benchmark/VerifyBlobKZGProofBatch(count=1)-20 585 2035749 ns/op Benchmark/VerifyBlobKZGProofBatch(count=2)-20 370 3227315 ns/op Benchmark/VerifyBlobKZGProofBatch(count=4)-20 243 4920444 ns/op Benchmark/VerifyBlobKZGProofBatch(count=8)-20 145 8207486 ns/op Benchmark/VerifyBlobKZGProofBatch(count=16)-20 82 14729597 ns/op Benchmark/VerifyBlobKZGProofBatch(count=32)-20 43 27678134 ns/op Benchmark/VerifyBlobKZGProofBatch(count=64)-20 21 53038206 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=1)-20 583 2039326 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=2)-20 538 2224141 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=4)-20 501 2383239 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=8)-20 451 2656815 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=16)-20 366 3253719 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=32)-20 187 6241685 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=64)-20 100 11882722 ns/op ``` For a better comparision of efficiency, with `GOMAXPROCS=1`: ``` Benchmark/BlobToKZGCommitment 26 43713460 ns/op Benchmark/ComputeKZGProof 25 45990877 ns/op Benchmark/ComputeBlobKZGProof 25 46085868 ns/op Benchmark/VerifyKZGProof 859 1390975 ns/op Benchmark/VerifyBlobKZGProof 585 2035813 ns/op Benchmark/VerifyBlobKZGProofBatch(count=1) 588 2035962 ns/op Benchmark/VerifyBlobKZGProofBatch(count=2) 355 3381252 ns/op Benchmark/VerifyBlobKZGProofBatch(count=4) 225 5297926 ns/op Benchmark/VerifyBlobKZGProofBatch(count=8) 135 8825641 ns/op Benchmark/VerifyBlobKZGProofBatch(count=16) 74 15600914 ns/op Benchmark/VerifyBlobKZGProofBatch(count=32) 39 29332908 ns/op Benchmark/VerifyBlobKZGProofBatch(count=64) 20 55860546 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=1) 584 2047389 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=2) 286 4179565 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=4) 141 8363224 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=8) 72 16752780 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=16) 34 33594562 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=32) 16 67546948 ns/op Benchmark/VerifyBlobKZGProofBatchPar(count=64) 8 134386234 ns/op ```
Benchmarks for C-KZG-4844 (Go bindings) ``` Benchmark/BlobToKZGCommitment-20 28 40685543 ns/op Benchmark/ComputeKZGProof-20 27 42668193 ns/op Benchmark/ComputeBlobKZGProof-20 27 42853123 ns/op Benchmark/VerifyKZGProof-20 1162 1020716 ns/op Benchmark/VerifyBlobKZGProof-20 742 1609853 ns/op Benchmark/VerifyBlobKZGProofBatch(count=1)-20 751 1596980 ns/op Benchmark/VerifyBlobKZGProofBatch(count=2)-20 452 2648943 ns/op Benchmark/VerifyBlobKZGProofBatch(count=4)-20 260 4590053 ns/op Benchmark/VerifyBlobKZGProofBatch(count=8)-20 141 8483093 ns/op Benchmark/VerifyBlobKZGProofBatch(count=16)-20 73 16198360 ns/op Benchmark/VerifyBlobKZGProofBatch(count=32)-20 37 31688225 ns/op Benchmark/VerifyBlobKZGProofBatch(count=64)-20 18 62790266 ns/op ```
sauliusgrigaitis commented 1 year ago

There is a third option - rust-kzg. It exposes the same C API as c-kzg-4844 so you can use the same Go bindings that are provided with c-kzg-4844. See here for instructions.

karalabe commented 1 year ago

@jtraglia The trusted setup file seems to be slightly different for the two libs: The Go version requires some Lagrange form of G1 points whereas the C does not. Any particular reason for that / not being able to calculate it internally?

sauliusgrigaitis commented 1 year ago

Conversion is expensive. But I'm in favor that all the libs should use exactly the same trusted setup file.

karalabe commented 1 year ago

The trusted setup file in the Go lib has 4096 G2 points but the C lib rejects that because it wants 65 G2 points. Which is correct?

sauliusgrigaitis commented 1 year ago

65 G2 points is more than enough.

jtraglia commented 1 year ago

Hey @karalabe, good question. For both presets, there should only be 65 G2 points. I suspect Go-KZG-4844 has 4096 because that's how the trusted setup was defined initially; about 4 months ago, this PR changed the count from 4096 to 65. Also, you should use the trusted setups (mainnet and minimal) from the consensus-specs.

lightclient commented 2 months ago

4844 shipped in March :confetti_ball: https://ethereum.org/en/roadmap/dencun/