Hi, it seems, that there is something wrong with the CI, that is displayed:
it's hugely inflated
it is not symmetric around the avg. score
While the inflated size could be justified, if it was defined somewhere, as to what kind of CI it is (in case it wouldn't be the standard 95% CI), the asymmetric shape doesn't make sense to me, at all.
As jmh provides the std-error in an aggregated field, already, I think this should also be visualized (if anything at all). I think 99% of the people, used to interpreting CIs at all, are used to seeing plain SEs (like jmh is providing) and can then derive their own CIs from it, intuitively - at whatever confidence-level they would like to see applied (probably the 95% CIs for most people, also = ~ +/- 1.96, assuming a normal distribution).
Hi, it seems, that there is something wrong with the CI, that is displayed:
While the inflated size could be justified, if it was defined somewhere, as to what kind of CI it is (in case it wouldn't be the standard 95% CI), the asymmetric shape doesn't make sense to me, at all.
example-data in jmh-visualizer.
I'm attaching my own visualization (which is much more ugly, but with correct error-bars): example-data with correct error-bars.
As jmh provides the std-error in an aggregated field, already, I think this should also be visualized (if anything at all). I think 99% of the people, used to interpreting CIs at all, are used to seeing plain SEs (like jmh is providing) and can then derive their own CIs from it, intuitively - at whatever confidence-level they would like to see applied (probably the 95% CIs for most people, also = ~ +/- 1.96, assuming a normal distribution).