search

ocaml-bench / sandmark

A benchmark suite for the OCaml compiler
The Unlicense
82 stars 40 forks source link

Include MPL benchmarks in the nightly pipeline #439

Closed Firobe closed 1 year ago

Firobe commented 1 year ago

Fix mpl benchmarks naming conventions and include it in the nightly runs. Additionally, document said conventions.

See https://github.com/ocaml-bench/sandmark/pull/404#issuecomment-1377033072

kayceesrk commented 1 year ago

When this PR is merged, it would be useful to check that the results do appear in the Sandmark Nightly UI. I hope we can remember to do this after the PR is merged. Otherwise, is there a quick way to check (in the CI) whether the additions will run and be visible in the Sandmark nightly UI?

shakthimaan commented 1 year ago

it would be useful to check that the results do appear in the Sandmark Nightly UI.

Sure.

is there a quick way to check (in the CI) whether the additions will run and be visible in the Sandmark nightly UI

These benchmarks are tagged with macro_bench and hence they are not run in the CI.

shakthimaan commented 1 year ago

LGTM!

punchagan commented 1 year ago

The results don't appear in the UI, since the serial version of the benchmarks is missing.

@Firobe as mentioned by KC here

The benchmark should have a serial version with the name `foo`.
The benchmark should have a parallel version with the name `foo_multicore`

So, the config files need to have serial runs for each of the MPL benchmarks, msort_ints, msort_strings, etc. (Again, as mentioned by KC in the comment referenced above, Parallel version running on 1 domain is not the same as the sequential version)

You can look at the mandelbrot6 and mandelbrot6_multicore benchmarks, as an example, if you need one. Thanks!