bheisler / criterion.rs

Statistics-driven benchmarking library for Rust
Apache License 2.0
4.28k stars 290 forks source link

how to call async functions during setup process? #748

Open pdeva opened 7 months ago

pdeva commented 7 months ago

i have code like that needs to read from S3, like this:

fn bench_s3(c: &mut Criterion) {
  let rt = tokio::runtime::Runtime::new().unwrap();
  let mut get_group = c.benchmark_group("s3_get");
    for (_, file) in files.iter().enumerate() {
        get_group.bench_with_input(
            BenchmarkId::new("single_file", file),
            file,
            |bencher, mybody| {
                bencher
                    .to_async(&rt)
                    .iter(|| async { s3_get_single(file).await });
            },
        );
    }
}

however, there needs to be a separate 'setup' part which actually PUTs those files to s3. and that portion will also be async. However, the bench_s3 method itself cannot be async. So where am I supposed to call the async setup method to setup my benchmarks?

shepmaster commented 6 months ago

You can use AsyncBencher::method.iter_custom. Something like:

.iter_custom(|| async { 
    s3_put().await;

    let start = Instant::now();
    s3_get_single(file).await;
    start.elapsed()
 });