Closed RubenVerborgh closed 9 years ago
The margin of errors are what make the difference here. The ones highlighted are the fastest discounting their margin of errors.
Thanks for explaining, but that doesn't explain everything I think. I subtracted the margin of errors:
So seems it should still be A & B.
Hmm, I'll review that bit in the code. Would you share the jsperf test you saw this on.
It was this one: http://jsperf.com/unknown-hash-key/2
For some insight into our process we sort based on mean
+ moe
but then run them through compare to perform a Mann-Whitney U test on them.
If you can reproduce this again after your run would you open your web console and do ui.benchmarks.map(function(b){return JSON.stringify(b.stats,null,2)})
then paste the output into a gist and report back.
Tried a few times but haven't been able to reproduce… I suppose the above numbers and percentages are insufficient to make this into a unit test?
Tried a few times but haven't been able to reproduce…
If you see it show up again then, no worries.
I suppose the above numbers and percentages are insufficient to make this into a unit test?
Right, we'd need the full stats record to be able to dig into it and see what's up.
A recent test gave me the following result:
while that should have been