search

bheisler / criterion.rs

Statistics-driven benchmarking library for Rust
Apache License 2.0
4.28k stars 290 forks source link

Sample size scaling for running in CI #747

Open tgross35 opened 7 months ago

tgross35 commented 7 months ago

Criterion alternative Divan claims to be able to run benchmarks in CI using "sample size scaling" to reduce timing noise: https://nikolaivazquez.com/blog/divan/#benchmark-in-ci.

Is this something that could be added to criterion?

FilipAndersson245 commented 3 months ago

Any update on this? It would be interesting if an alternative sampling mode simular to divian could be implemented? I think being able to run benches in noisy environment like CI would enable more usecases for Criterion. Divan bases a lot if its design on this paper