bheisler / criterion.rs

Statistics-driven benchmarking library for Rust
Apache License 2.0
4.28k stars 290 forks source link

Custom Iter with async setup and non-async benched value #762

Open Sapp00 opened 4 months ago

Sapp00 commented 4 months ago

Hello, I have the following bench:

pub fn criterion_benchmark_calc_path(c: &mut Criterion) {
    let runtime = Builder::new_multi_thread().enable_all().build().unwrap();

    c.bench_function("calc_path", |b| {
        b.to_async(&runtime).iter_custom(|iters| async{
                let mut olist = PathManager::get_list().await;

                let start = Instant::now();
                let _ = pm.calculate();

                start.elapsed()
        })
    });
}

When I run this bench, it returns 0 as time, but this doesnt make sense. When I add some noise with sleep, it returns only the sleep duration. Is there any known limitation to the async bencher?