search

joshuaulrich / microbenchmark

Infrastructure to accurately measure and compare the execution time of R expressions
Other
95 stars 24 forks source link

Enhancement : Test for significance of execution time differences #45

Open jeromepcollet opened 1 year ago

jeromepcollet commented 1 year ago

Add significance test of time difference

In the summary of a benchmark, it would be useful to have a test of significance of the time differences. It is possible with the following code : kruskal.test(time ~ expr, data = bench) I propose Kruskal test because it works for any number of groups.

tinygreen commented 10 months ago

Try the already implemented Tukey test which is included in the output if the multcomp package is installed. Note issue #43.

jay-sf commented 6 months ago

I think this Stack Overflow answer shows that the cld column actually shows groups with no significant differences (rather than a ranking).