search

jzillmann / gradle-jmh-report

A Gradle plugin building a visual report for JMH benchmarks
Apache License 2.0
46 stars 3 forks source link

Jenkins and multi-jmh report visualization #9

Open sebersole opened 3 years ago

sebersole commented 3 years ago

The JMH Visualizer is able to visualize multiple report files. Can this be done using the Jekins plugin?

I have a project that runs some JMH tests against my latest project code and against a set, older version. So the single project produces multiple report files.

jzillmann commented 3 years ago

@sebersole Did you try https://github.com/jenkinsci/jmh-report-plugin ?

sebersole commented 3 years ago

Yes I use that plugin. I can ask the same question there if you like. But I still do not see anywhere on that page where it discusses comparing 2 benchmarks

jzillmann commented 3 years ago

Ok, think I might understand it now...

The main web app (https://github.com/jzillmann/jmh-visualizer) kind of supports multiple files/reports and they are stacked vertically, means each file represents a run and multiple runs can be compared against each other.

You want exactly that from a single Jenkins execution. Multiple reports, compared against each other. (The multiple files shouldn't be merged to a single report/run - which is what https://github.com/jenkinsci/jmh-report-plugin/pull/5 is about)

Well, I think its doable but is really something for the Jenkins plugin (since the plugin isn't tight to the Gradle plugin, it just consumes the regular JMH json file. Also the Jenkins project level report wouldn't make much sense to you since you always track each version in a single execution ?