search

jzillmann / jmh-visualizer

Visually explore your JMH Benchmarks
https://jmh.morethan.io
GNU Affero General Public License v3.0
294 stars 22 forks source link

Show nano fractions #21

Open chrisgleissner opened 5 years ago

chrisgleissner commented 5 years ago

My measurements have very small nano results, e.g. 4.123456ns. The visualizer omits the fractional part and renders this as 4, thus making it impossible to view small benchmark variations.

You can see this behaviour in the GetterBenchmark at https://jmh.morethan.io/?source=https://raw.githubusercontent.com/chrisgleissner/benchmarks/master/jmh-result.json

The rendered measurements for the direct and lambdaMetaFactoryForGetter benchmark tests both appear as 4ns when their raw values from the Json files differ.

Would it be possible to show at least one fractional digit for nano measurements? For example, a raw value of 4.123456ns from the json could be rendered as 4.1.

If you are concerned about the visual overhead this feature would add, I see two possible workarounds:

Thanks

chrisgleissner commented 5 years ago

Hello,

It sums many samples, then divides by the number of samples.

For example, if you have 1000 measurements that take a combined 1.1microsecond, jmh will divide 1100ns by 1000 and report 1.1ns.

The feature of showing at least one fractional unit for measurements of less than 100ns would be great.

Kind Regards Christian

On Mon, 21 Jan 2019, 11:13 Idan Sheinberg <notifications@github.com wrote:

@chrisgleissner https://github.com/chrisgleissner how exactly does the JVM measure sub nanoseconds duration?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jzillmann/jmh-visualizer/issues/21#issuecomment-456037658, or mute the thread https://github.com/notifications/unsubscribe-auth/ADyQe7IKNiJy0BERvlqXU7zWt8c1xkzzks5vFaDHgaJpZM4aKaEp .

chrisgleissner commented 5 years ago

I've now switched over from measuring throughput instead of average time, and this resolved my issue for very short operations in the nanosecond region. These now result in large values which allow for differentiating results.

It may still be worth changing the number rendering since the visualization switches from "no fractional digits" to "fractional digits" as a value drops below "1". For example, 1.0001 gets rendered as 1 whereas 0.99999 gets rendered with all fractional digits. This makes it hard to see differences where benchmark variations hover around 1.

Also, I wanted to thank you for your fantastic visualizer. Beautiful rendering and very fast.

jzillmann commented 5 years ago

@chrisgleissner Thank you for your appreciation. Yeah, with the rounding i went back and forth... Both rounding and not rounding can be disturbing, depending if your numbers are big or small. Right now there is even a auto behaviour which stops rounding if a number is below 1, but i found this feature not very robust.

Anyway, i was thinking about introducing a global configuration option to turn rounding on and of (could be flipped via url parameter or through menu). Any thoughts on that ?

chrisgleissner commented 5 years ago

Hi Johannes,

Thanks for your time. Yes, the global flag sounds like a very good idea.

Thanks Christian

On Wed, 1 May 2019, 10:17 Johannes Zillmann, notifications@github.com wrote:

@chrisgleissner https://github.com/chrisgleissner Thank you for your appreciation. Yeah, with the rounding i went back and forth... Both rounding and not rounding can be disturbing, depending if your numbers are big or small. Right now there is even a auto behaviour which stops rounding if a number is below 1, but i found this feature not very robust.

Anyway, i was thinking about introducing a global configuration option to turn rounding on and of (could be flipped via url parameter or through menu). Any thoughts on that ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jzillmann/jmh-visualizer/issues/21#issuecomment-488241681, or mute the thread https://github.com/notifications/unsubscribe-auth/AA6JA6ZRN7BYIZ3WEA5K66TPTFN23ANCNFSM4GRJUEUQ .

willhains commented 2 years ago

Both rounding and not rounding can be disturbing, depending if your numbers are big or small.

I think the goal here should be to get a minimum number of significant digits. This would work for both large and small numbers. As an example, let's say the target significant digits is four:

Actual Value Rounded Display Value
0.123456789 0.1235
12.3456789 12.35
123,456.789 123,457