jzillmann / gradle-jmh-report

A Gradle plugin building a visual report for JMH benchmarks
Apache License 2.0
46 stars 3 forks source link

Add params as label qualifier #2

Closed ben-manes closed 7 years ago

ben-manes commented 7 years ago

Thanks for this plugin! I was using jmh-charts, which had this same problem.

In benchmarks that use @Param the labels should qualify what is being tested. Otherwise the chart is not clear. See for example ComputeBenchmark which produces the chart below.

image

In jmh-charts that would look something like, image

jzillmann commented 7 years ago

Thanks @ben-manes. Supporting params is on my near term list!

jzillmann commented 7 years ago

@ben-manes I didn't fix it in the gradle plugin yet, but in the online visualizer: http://jmh.morethan.io Looks like this:

computebenchmark

The plan to make use of that visualizer in the next version of the gradle plugin. So any feedback on it will flow into the gradle plugin as well!

ben-manes commented 7 years ago

That looks very nice, thanks!

Perhaps the param value(s) should be bold and the bars given a different color? Alternatively, it might look nicer if showing the param name was optional and just the value(s) was the item's label. Then group by method with the name on the outer bracket, e.g. a nested y-label.

The thrpt ops/s could be the x-label and the chart title the benchmark name. Also the x values could be shortened with a magnitude label, e.g. 35M, 70M, ...

I think those changes would declutter the chart and make items standout more.

jzillmann commented 7 years ago

Does that match your expectation ?

computebenchmarkv2
ben-manes commented 7 years ago

That's great. Can add a label and use short numerical values on the x axis?

jzillmann commented 7 years ago

For the x-axis tick shortening i created a separate issue: https://github.com/jzillmann/jmh-visualizer/issues/2

What you mean by

add a label

?

ben-manes commented 7 years ago

The benchmark type, e.g. throughput

jzillmann commented 7 years ago

Ok, i pushed a new version online (http://jmh.morethan.io). Your caffeine suite now looks much better i think since it was using params heavily....

Not sure if you can decipher that, but FYI, this is how i'm handling the params:

  1. no param => standard
  2. one param, single method => convert to (0)
  3. one param, multi methods => Bar per param Value
  4. two params, single methods => convert to (2)
  5. two+ params, multi methods => combine params & convert to (2)
  6. three+ params, single methods => combine params & convert to (0)

If you have any more requests one the online tool, please use https://github.com/jzillmann/jmh-visualizer/issues.

Once i brought the online version into the gradle plugin, i will close this issue...

ben-manes commented 7 years ago

Thanks!

jzillmann commented 7 years ago

This is now released as 0.2.0!

ben-manes commented 7 years ago

Thanks!