jzillmann / jmh-visualizer

Visually explore your JMH Benchmarks
https://jmh.morethan.io
GNU Affero General Public License v3.0
294 stars 22 forks source link

Data isn't visualized for small values #6

Closed BrainStone closed 7 years ago

BrainStone commented 7 years ago

I typically prefer the sample benchmark over the throughput one. However results are usually around below 1 ms/op. The graph then shows all values as zeros. Decimal values should be supportted in graphing!

Example json: https://gist.github.com/BrainStone/1ec997aaf3cbd07ac925bd6e66d9bcfb

jzillmann commented 7 years ago

Ya, thats true i added a rounding since for my benchmarks the decimals were just annoying ;) Question. Would it make sense in your case to switch to the next lower OutputTimeUnit, like microseconds ?

If not i was thinking about something like when the values are bigger then a certain threshold (lets say 10) then do the rounding otherwise don't.

BrainStone commented 7 years ago

Generally speaking, why round at all? These small differences might be important and people might not have the possibility or the time (a proper benchmark could take days) to redo the benchmark with a different scale.

jzillmann commented 7 years ago

For me the main benefit is of having the visualizer is cutting through the noise and seeing the most important thing as easy as possible by not being distracted by un-important things! For the scales i worked with so far, decimals were clearly unimportant. And i understand that the situation is different for you.

Just wanna check if you are aware on changing the unit and if so, what are your reasons!

BrainStone commented 7 years ago

I understand that. But regardless if you want to round I'd recommend rounding to sig figs. Like on the graph 2 sig figs should be sufficient and when hovering over the data 4 are a good choice I think.

To illustrate, here is a small table:

Original value 2 sig figs (labels on the graph) 4 sig figs (when hovering)
123456 120000 123500
123.456 120 123.5
12.3456 12 12.35
1.23456 1.2 1.235
0.123456 0.12 0.1235
0.000123456 0.00012 0.0001235

Or in scientific notation (which really illustrates the way of rounding):

Original value 2 sig figs (labels on the graph) 4 sig figs (when hovering)
1.23456e5 1.2e5 1.235e5
1.23456e2 1.2e2 1.235e2
1.23456e1 1.2e1 1.235e1
1.23456e0 1.2e0 1.235e0
1.23456e-1 1.2e-1 1.235e-1
1.23456e-4 1.2e-4 1.235e-4

There should be methods somewhere in the internet that do that rounding.

jzillmann commented 7 years ago

Along with the 0.5.1 version i disabled rounding for datasets where each score was below a certain threshold. Please give it a try and let me know how you like it!