Thanks so much for creating Codspeed! For libraries like mine with a strong performance focus and a reliance on benchmarking, this fills a gap in my CI pipeline that I've had to fill with manual benchmark runs.
I'm following the instructions here to switch to the Codspeed Criterion compatibility layer but I've ran into two incompatibilities that are preventing me from continuing (I'm using criterion 0.5).
In case it is helpful, the PR I have open to switch to Codspeed is here: https://github.com/sdd/kiddo/pull/98. (EDIT: now that I've raised a PR on codspeed-rust to fix the problem, that linked PR has been updated to use the patched codspeed-rust on the branch submitted for that PR)
There are two issues:
My benchmarks contain functions that have a parameter of type BenchmarkGroup<WallTime> . Whilst I can change this to BenchmarkGroup from the compatibility layer, this breaks things when trying to run Criterion.
I'm also using criterion's PlotConfiguration to set a logarithmic scale. I've got a couple of lines like this, in a few places:
let plot_config = PlotConfiguration::default().summary_scale(AxisScale::Logarithmic);
group.plot_config(plot_config);
The BenchmarkGroup from the compatibility layer does not implement the plot_config method. I can comment this out to get Codspeed working, but then I'll lose that functionality with Criterion.
Let me know if I can help in any way towards getting this working. Happy to submit a PR if you can briefly point me in the right direction regarding the changes I'd need to make. PR submitted.
Hi Arthur,
Thanks so much for creating Codspeed! For libraries like mine with a strong performance focus and a reliance on benchmarking, this fills a gap in my CI pipeline that I've had to fill with manual benchmark runs.
I'm following the instructions here to switch to the Codspeed Criterion compatibility layer but I've ran into two incompatibilities that are preventing me from continuing (I'm using criterion 0.5).
In case it is helpful, the PR I have open to switch to Codspeed is here: https://github.com/sdd/kiddo/pull/98. (EDIT: now that I've raised a PR on codspeed-rust to fix the problem, that linked PR has been updated to use the patched codspeed-rust on the branch submitted for that PR)
There are two issues:
My benchmarks contain functions that have a parameter of type
BenchmarkGroup<WallTime>
. Whilst I can change this toBenchmarkGroup
from the compatibility layer, this breaks things when trying to run Criterion.I'm also using criterion's
PlotConfiguration
to set a logarithmic scale. I've got a couple of lines like this, in a few places:The
BenchmarkGroup
from the compatibility layer does not implement theplot_config
method. I can comment this out to get Codspeed working, but then I'll lose that functionality with Criterion.Let me know if I can help in any way towards getting this working. Happy to submit a PR if you can briefly point me in the right direction regarding the changes I'd need to make.PR submitted.Thanks!