1) Key generation benchmarking uses a cryptographic rng, this leads to wildly fluctuating benches due to the bottleneck of entropy generation and is basically worthless. Temporarily should probably be removed.
It needs to be replaced with a non-cryptographic rng to get accurate readings, this should probably be something like fastrand or perhaps XXHash with deterministic output.
2) Setup a github action using the current Criterion benchmark harness that can report any regressions on PR's.
mberry
commented
1 year ago
Two-part issue:
1) Key generation benchmarking uses a cryptographic rng, this leads to wildly fluctuating benches due to the bottleneck of entropy generation and is basically worthless. Temporarily should probably be removed. It needs to be replaced with a non-cryptographic rng to get accurate readings, this should probably be something like fastrand or perhaps XXHash with deterministic output.
2) Setup a github action using the current Criterion benchmark harness that can report any regressions on PR's.
https://docs.rs/fastrand/latest/fastrand/
https://github.com/DoumanAsh/xxhash-rust
https://github.com/benchmark-action/github-action-benchmark