CI: use benchmark groups when reporting results

celerity / celerity-runtime

High-level C++ for Accelerator Clusters

https://celerity.github.io

MIT License

139 stars 18 forks source link

CI: use benchmark groups when reporting results #222

Closed PeterTh closed 10 months ago

PeterTh commented 10 months ago

Also only add benchmarks to the charts if there is a notable difference
Tweak thresholds to be a bit less sensitive
Improve script documentation

Closes https://github.com/celerity/meta/issues/47.

github-actions[bot] commented 10 months ago

Check-perf-impact results: (b9a1eb21ba2b25ca13f86fa32e06f70b)

:warning: Significant slowdown in some microbenchmark results: normalizing randomized box sets - 2d / large, native
:rocket: Significant speedup in some microbenchmark results: 20 individual benchmarks affected

Relative execution time per category: (mean of relative medians)

command-graph : 0.92x
graph-nodes : 1.02x
grid : 1.06x
scheduler : 0.87x :rocket:
system : 0.54x :rocket:
task-graph : 0.99x

PeterTh commented 10 months ago

The data above is fake (or rather, from a run on a completely different system from our canonical benchmark system), I just included it for testing. I will now remove that commit.

github-actions[bot] commented 10 months ago

Check-perf-impact results: (b003273516680ef3e6ca0110b3678f5e)

:question: No new benchmark data submitted. :question:
Please re-run the microbenchmarks and include the results if your commit could potentially affect performance.