search

csgillespie / benchmarkme

Crowd sourced benchmarking
https://csgillespie.github.io/benchmarkme/
40 stars 13 forks source link

Bug in benchmark::rank_results #16

Closed XiaomoWu closed 5 years ago

XiaomoWu commented 6 years ago

The number of rank is larger than the total number of sample, after running the rank_results function. Here is how to reproduce the bug:

library(benchmarkme)
res_cpu <- benchmark_std(runs = 3)
rank_results(res_cpu)

# You are ranked 70 out of 45 machines. (the number may vary across different machines)

A closer look at the rank_results function reveals that the bug lies in the last few lines:

    if (verbose)
        message("You are ranked ", ben_rank, " out of ", nrow(results),
            " machines.")

where nrow(results) should be nrow(pas_res).

Another question I want to confirm is that does the past performance dataset benchmarkmeData::past_results change/accumulate as time passes by? If so, then perhaps I may yield different benchmarks for the same machine?

Thank you. Yu

csgillespie commented 5 years ago

Sorry for the very long delay.

Thanks