sharkdp / hyperfine

A command-line benchmarking tool
Apache License 2.0
21.67k stars 349 forks source link

Question about hyperfine benchmark results #665

Open Pavel-Durov opened 1 year ago

Pavel-Durov commented 1 year ago

I was looking at hyperfine documentation but I couldn't find anything related to the the output format.

How do I read hyperfine benchmark results?

Example:

  Time (mean ± σ):      14.9 ms ±   0.7 ms    [User: 12.5 ms, System: 2.3 ms]
  Range (min … max):    14.0 ms …  21.3 ms    187 runs

min .. max - minimum and maximum running times User - time spent in user space, amount of time spent on the program itself System - time spent in system space, amount of time spent on OS functions related to the program runs - total count of benchmark runs

What's mean ± σ? Standard deviation?

Pavel-Durov commented 1 year ago

The way I interpret it:

mean - is the average execution time of the benchmarks ± σ - is the standard deviation representing the amount of variation from the mean

Is it somehow correct?

sharkdp commented 1 year ago

Please see #443

sharkdp commented 1 year ago

@Pavel-Durov does this help?