CodSpeedHQ / codspeed-rust

Rust crates to create CodSpeed benchmarks
https://codspeed.io
Apache License 2.0
23 stars 9 forks source link

Support `criterion` benchmark group displays #20

Open EYHN opened 1 year ago

EYHN commented 1 year ago

First of all I would like to thank you for making such a good benchmark action, which is almost the only well-functioning and well-maintained benchmark action on github. 🙏 🎉

I have a benchmark that uses criterion benchmark_group to compare multiple functions, for example I have the following benchmark code:

let mut group = c.benchmark_group("Map Insert");
group.bench_function("HashMap", |b| {
    ...
});
group.bench_function("BTreeMap", |b| {
    ...
});

When I run the benchmark locally, I can see in the console that the benchmark's name is a combination of the group's name and the function's name.

Map Insert/HashMap      time:   [275.12 ns 284.51 ns 293.79 ns]
Map Insert/BTreeMap     time:   [418.15 ns 424.73 ns 431.82 ns]

However, when running in codspeed, the result is only the function name, without the group name, which makes it difficult to compare.

图片
ctz commented 3 weeks ago

Having just got started with codspeed (having learned about it at eurorust this week) I have the same experience.

eg, compare https://codspeed.io/ctz/graviola/benchmarks and https://jbp.io/graviola/reports/x86_64/report/index.html (from the same set of benchmark code)