search

localvoid / vdom-benchmark

Virtual DOM diff/patch benchmark
BSD 2-Clause "Simplified" License
49 stars 10 forks source link

Results are not very consistent #3

Closed Bobris closed 9 years ago

Bobris commented 9 years ago

I am unsure that average of 3 runs is best metric (IMHO fastest of 3 runs could make more consistent results). Also why to include init render time - especially on that you can see inconsistency, when all in same category should be identical...

localvoid commented 9 years ago

It is terrible metric, I just wish that there is something like Haskell's Criterion library in Dart or any basic stats library, and I am really lazy to do anything more than that :)

I've included init render time because of super-fast mithriljs rendering (but it seems that they've talked about the library size, and how fast it is loaded over the network).

AlexGalays commented 9 years ago

From what I've seen, the results are inconsistent because of GC running at various points in time; Libraries creating more garbage will be punished more heavily, but that's expected.

Maybe increase the number of iteration to 10?

Bobris commented 9 years ago

It is probably best what can be done in reasonable time.