Open shakthimaan opened 2 years ago
@shakthimaan Would the configure field not be sufficient for this?
Would the configure field not be sufficient for this?
For a single run, yes. At present, the unit of operation is a benchmark run. We do not have the notion of an experiment
.
It will be useful to describe an experiment with multiple test configuration options, run benchmarks, and compare the results.
Compiler developers would like to observe metric changes when passing
-O2
and-O3
configure options to the compiler when building the benchmarks.Reference: https://github.com/ocaml-bench/sandmark-nightly-config/pull/5#issuecomment-1185758083