In the Rust portion of the repo under benches, we should implement criterion in a way that allows a higher warmup of the CPU (3s to 5s) before sampling. In addition, we should increase the sample size from 100 -> 150 and bootstrap samples to 100,000 to have a better idea of deviations.
rust/protocol/benches/session.rs
// Needed for Criterion Warmup
use std::time::Duration;
// Old Benchmark Code Here //
// remove the following
//criterion_group!(benches, session_encrypt, session_encrypt_decrypt);
criterion_group!(
name = benches;
config = Criterion::default()
.warm_up_time(Duration::from_secs(5))
.measurement_time(Duration::from_secs(10))
// degree of noise to ignore in measurements, here 1%
.noise_threshold(0.01)
// likelihood of noise registering as difference, here 5%
.significance_level(0.05)
// likelihood of capturing the true runtime, here 95%
.confidence_level(0.95)
// total number of bootstrap resamples, higher is less noisy but slower
.nresamples(100_000)
// total samples to collect within the set measurement time
.sample_size(150);
targets = session_encrypt, session_encrypt_decrypt
);
criterion_main!(benches);
In the Rust portion of the repo under benches, we should implement
criterion
in a way that allows a higher warmup of the CPU (3s to 5s) before sampling. In addition, we should increase the sample size from 100 -> 150 and bootstrap samples to 100,000 to have a better idea of deviations.rust/protocol/benches/session.rs