nvzqz / divan

Fast and simple benchmarking for Rust projects
https://nikolaivazquez.com/blog/divan/
Apache License 2.0
924 stars 26 forks source link

RFC: Ability to add post-result counters after #34

Open vlovich opened 10 months ago

vlovich commented 10 months ago

I was benchmarking a cache library and would like to have the output contain a result of the cache hit rate so that I can understand the benchmark performance across multiple dimensions. I'm thinking something like:

let mut hits = 0;
let mut misses = 0;
b
.output_counter(|| counter::Ratio::new("Hit performance", hits, hits + misses))
.bench(|| {
   // adjust hits & misses
})

It's a little bit complicated since hits & misses borrowing is not going to work like that without unsafe / RefCell / atomics. An alternative approach could be for .bench to return something:

let mut hits = 0;
let mut misses = 0;
b.bench(|| {
   // adjust hits & misses
  (hits, misses)
}).output_counter(|(hits, misses)| counter::Ratio::new("Hit performance", hits, hits + misses))

This is all a bit hand-wavy as I don't have the exact details in mind about the API (open to suggestions), but curious if there's any interest in me adding something like this.

nvzqz commented 10 months ago

I haven't had time to think much about custom counters (planned), so I'm curious how you think Ratio should be displayed?

I know how Bencher::output_counter could be implemented, so that could be done relatively soon. Currently I'm prioritizing #18, async, and runtime arguments (as opposed to consts).

vlovich commented 10 months ago

An example of displaying it would be something like

      56.14 ms
      1.781 Mitem/s
      0.86 Hit performance

I have no strong feelings on the matter though.

I'm happy to help put up a PR if you'd like any help (& if you have a specific way you want it implemented just let me know).