Currently we are using the running script (https://github.com/mmtk/running/tree/rust-mmtk) for configuring runs and executing benchmarks. However, there are some limitations of the running script, such as:
It is designed for executing Java benchmarks, we probably cant use it for Javascript benchmarking.
It assumes heap size for each benchmark and we can't set heap size for each build.
It assumes the same iteration/invocation for all the benchmarks. We may want to run certain benchmarks with more invocations/iterations to get more reliable results.
We may consider using ReBench (https://github.com/smarr/ReBench). That is used by SOM across multiple SOM implementation. We need to investigate to see if ReBench is suitable.
Currently we are using the
running
script (https://github.com/mmtk/running/tree/rust-mmtk) for configuring runs and executing benchmarks. However, there are some limitations of therunning
script, such as:We may consider using ReBench (https://github.com/smarr/ReBench). That is used by SOM across multiple SOM implementation. We need to investigate to see if ReBench is suitable.