search

briansmith / crypto-bench

Benchmarks for crypto libraries (in Rust, or with Rust bindings)
70 stars 11 forks source link

Use stable Rust instead of nightly Rust #41

Closed briansmith closed 4 years ago

briansmith commented 6 years ago

Benchmarks on nightly Rust aren't as useful as benchmarks on stable Rust would be since most users want to use stable Rust (at least, according to my experience).

I would be happy to switch to either https://crates.io/crates/bencher or https://crates.io/crates/criterion; see https://www.reddit.com/r/rust/comments/8tplwr/what_is_the_standard_way_to_run_benchmarks_in/ for more discussion. One problem may be "black box" support, or the lack of it.

briansmith commented 5 years ago

The only realistic solution right now seems to be to switch to criterion.rs. See https://bheisler.github.io/post/benchmarking-with-criterion-rs/ and https://llogiq.github.io/2018/05/18/criterion.html.

hauleth commented 5 years ago

I am using Criterion in benching Octavo and it is quite pleasant. Also as soon as bheisler/criterion.rs#130 lands it will be perfect solution as it will be able to measure CPU cycles instead of wall time, as this is metric commonly used in crypto benching.