search

ocaml-bench / sandmark

A benchmark suite for the OCaml compiler
The Unlicense
82 stars 38 forks source link

Makefile variable to pass configure options when building benchmarks #375

Open shakthimaan opened 2 years ago

shakthimaan commented 2 years ago

Compiler developers would like to observe metric changes when passing -O2 and -O3 configure options to the compiler when building the benchmarks.

Reference: https://github.com/ocaml-bench/sandmark-nightly-config/pull/5#issuecomment-1185758083

punchagan commented 8 months ago

@shakthimaan Would the configure field not be sufficient for this?

shakthimaan commented 8 months ago

Would the configure field not be sufficient for this?

For a single run, yes. At present, the unit of operation is a benchmark run. We do not have the notion of an experiment.

It will be useful to describe an experiment with multiple test configuration options, run benchmarks, and compare the results.