search

atemerev / skynet

Skynet 1M threads microbenchmark
MIT License
1.04k stars 122 forks source link

What, if anything, have you done to thwart CPU frequency scaling? #30

Open jlouis opened 8 years ago

jlouis commented 8 years ago

You need to make sure your CPU is running at full clock speed when starting the test. Otherwise, part of the test is a scaling of the CPU frequency, which isn't fair.

If you run the test 20 times for each application and measure all the results, what are the variance of the measurements? If you use something like Haskell's criterion or my own eministat to analyze the results, are they stable against outlier variance?

ChristopherKing42 commented 8 years ago

Haskell's code is able to detect variance: https://travis-ci.org/ChristopherKing42/skynet/builds/109503734#L460