search

sigp / lighthouse

Ethereum consensus client in Rust
https://lighthouse.sigmaprime.io/
Apache License 2.0
2.91k stars 738 forks source link

Benchmark parallel kzg verification on testnets #4242

Open pawanjay176 opened 1 year ago

pawanjay176 commented 1 year ago

Description

Related to https://github.com/ethereum/c-kzg-4844/pull/192

Currently, we use the verify_blob_kzg_proof_batch function when trying to verify multiple blobs. However, it seems like doing parallel verification with a rayon threadpool for verify_blob_kzg_proof is faster. We should test the parallel verify_blob_kzg_proof against verify_blob_kzg_proof_batch in lighthouse to check if it gives us similar speedups when running lighthouse.

realbigsean commented 1 year ago

I think @ethDreamer has done investigation here and has seen a speed up of about 5%. Was that across batches of 32?

ethDreamer commented 11 months ago

it was batches of 64 actually but yes.

some details

pawanjay176 commented 11 months ago

Your linked PR is super nice and tidy. Been meaning to do this for a long time. Thanks for that.

From your comment on the PR, I understood that calling verify_kzg_for_blob_list for all blobs in a batch instead of calling it in a loop for each block in a batch is 32x faster.

Wondering if you have also tried changing verify_kzg_for_blob_list implementation to call verify_kzg_for_blob in a rayon thread pool. From the linked PR https://github.com/ethereum/c-kzg-4844/pull/192 , it seems like the rayon implementation was giving ~2x improvement for 64 batch size in isolation.