AndrasKovacs / normalization-bench

Lambda normalization and conversion checking benchmarks for various implementations
MIT License
55 stars 6 forks source link

JVM results are unreliable #7

Closed newca12 closed 4 years ago

newca12 commented 4 years ago

Each run give very different results, for example Nat 5M conversion min 121.7 max 221.3 Nat 5M normalization min 270.85 max 362.75 Nat 10M conversion min 643.75 max 817.05

This is expected, you cannot reliabily bench a JVM program the same way you do with haskell or OCaml. One way to fix this is to use a tool like JMH.