dart-lang / benchmark_harness

The official benchmark harness for Dart
https://pub.dev/packages/benchmark_harness
BSD 3-Clause "New" or "Revised" License
94 stars 26 forks source link

add printout of "time per unit" in nanoseconds #48

Open tatumizer opened 4 years ago

tatumizer commented 4 years ago

Current printout (time to run 10 times, in microseconds) might be good only for relative measurements and regression testing, but not for evaluation of the performance in absolute terms. E.g. when we benchmark "list copy" operation using 100-element lists, we can tell how List.of compares with toList, but it tells nothing about actual performance. If we had printout in nanoseconds per 1 copied list item, we can compare it against common sense. Common sense, in this case, says that if we have a reading above 2 nanoseconds/item on a 3+Ghz processor, there's room for improvement. And if we measure 100 nanoseconds per item (instead of 2) then there's a lot of room for improvement. Etc.

vaind commented 3 years ago

Hope it's fine to post this here, if not, please remove....

I've been missing some features in benchmark_harness as well, which led me to develop an alternative. After having it in-code in some projects, I've now decided to publish it for others, check out https://pub.dev/packages/benchmarking