General question

[d6u]

Awesome experiment. I accidentally noticed this. and found it provided information I never know before. After reading the chart and README, I have a few questions:

1. When I look at the chart, the differences after 99% percentile is very obvious. But does it only counts as 1% of all cases? Should we pay more attention to cases below 99 percentile? Looks like on average, OCaml doesn't performance as well as other languages below 90 percentile. Maybe I misunderstood the purpose of those tests.
2. The small num in chart is not entirely obvious. Again, I could be wrong about the purpose. Could logarithmic scale showcase the data below 90 percentile better? I doesn't seem like hdrhistogram have a logarithmic scale option though.

[spion]

The test is meant to check the soft-realtime capabilities. Latency needs in these cases will vary depending on the application

I'll start with two use cases to illustrate:

• music jamming service - latencies should be in the < 15ms range for this to work (!)
• games - slightly less sensitive than music, but I'd say things start to suck above 70-80ms

For both of these cases, > 99% is very important. Even if one of 100 actions was delayed by 100ms because of the garbage collector, that would still be unacceptable in a few-minute gaming (or jaming) session.

On the other hand, absolute performance isn't too important. Whats important is that its predictable and satisfactory - as long as the delays are small enough.

There is also the inherent network latency that needs to be added, and for non-local networks thats easily in the 10ms+ range: https://wondernetwork.com/pings/London - so in those cases we get diminishing returns for small service latencies (the difference between 6ms and 3ms is actually the difference between 16ms and 13 ms :D)

[d6u]

Thanks! Now I understand those tests.

[spion]

I'll try and add some of the explanation in the readme.