Multicore parallel benchmarks support for 4.14

shakthimaan commented 2 years ago

The domainslib4.x branch with dune.2.9.0 is used to build the parallel benchmarks in Sandmark to support 4.14 using the following commands in the CI:

 TAG='"run_in_ci"' make multicore_parallel_run_config_filtered.json
 TAG='"macro_bench"' make multicore_parallel_run_config_filtered_filtered.json
 OPT_WAIT=0 USE_SYS_DUNE_HACK=1 BUILD_BENCH_TARGET=multibench_parallel RUN_CONFIG_JSON=multicore_parallel_run_config_filtered_filtered_2domains.json make ocaml-versions/4.14.0.bench

At present it fails with the following error messages, and the benchmarks need to be updated to support both 4.x and 5.x branches:

opam exec --switch 4.14.0 -- rungen _build/4.14.0_1 multicore_parallel_run_config_filtered_filtered_2domains.json > runs_dune.inc
opam exec --switch 4.14.0 -- dune build --profile=release --workspace=ocaml-versions/.workspace.4.14.0 @multibench_parallel;
File "benchmarks/multicore-lockfree/bag-dune.inc", line 4, characters 12-20:
4 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/bag-dune.inc", line 9, characters 12-20:
9 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/hash-dune.inc", line 4, characters 12-20:
4 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/hash-dune.inc", line 9, characters 12-20:
9 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/list-dune.inc", line 4, characters 12-20:
4 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/list-dune.inc", line 9, characters 12-20:
9 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/msqueue-dune.inc", line 4, characters 12-20:
4 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/msqueue-dune.inc", line 9, characters 12-20:
9 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/wsqueue-dune.inc", line 4, characters 12-20:
4 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/multicore-lockfree/wsqueue-dune.inc", line 9, characters 12-20:
9 |     (libraries lockfree))
                ^^^^^^^^
Error: Library "lockfree" not found.
Hint: try:
  dune external-lib-deps --missing --profile release --workspace ocaml-versions/.workspace.4.14.0 @multibench_parallel
File "benchmarks/simple-tests/alloc_multicore.ml", line 17, characters 9-21:
17 |         (Domain.spawn (fun () -> for _ = 0 to w do
              ^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/simple-tests/pingpong_multicore.ml", line 26, characters 66-78:
26 | let intermediate_consumers = Array.init (num_domains-2) (fun i -> Domain.spawn (worker queues.(i) queues.(i+1)))
                                                                       ^^^^^^^^^^^^
Error: Unbound module Domain
         gcc benchmarks/multicore-gcroots/globroots/globrootsprim.o [4.14.0_1] (exit 1)
(cd _build/4.14.0_1/benchmarks/multicore-gcroots/globroots && /usr/bin/gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_FILE_OFFSET_BITS=64 -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -g -I /home/shakthi/testing/custom-4.14-variant-build/sandmark/_opam/4.14.0/lib/ocaml -o globrootsprim.o -c globrootsprim.c)
globrootsprim.c:22:10: fatal error: caml/shared_heap.h: No such file or directory
   22 | #include "caml/shared_heap.h"
      |          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
File "benchmarks/multicore-gcroots/globroots/globroots.ml", line 29, characters 21-39:
29 |   let random_state = Domain.DLS.new_key Random.State.make_self_init
                          ^^^^^^^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-structures/test_queue_parallel.ml", line 7, characters 4-16:
7 |     Domain.spawn (fun _ ->
        ^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-structures/test_spsc_queue_parallel.ml", line 7, characters 4-16:
7 |     Domain.spawn (fun _ ->
        ^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-structures/test_spsc_queue_pingpong_parallel.ml", line 15, characters 15-31:
15 |     | false -> Domain.cpu_relax(); spinning_enqueue q m
                    ^^^^^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-structures/test_stack_parallel.ml", line 7, characters 2-14:
7 |   Domain.spawn (fun _ ->
      ^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-minilight/parallel/camera.ml", line 13, characters 8-26:
13 | let k = Domain.DLS.new_key Random.State.make_self_init
             ^^^^^^^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-numerical/LU_decomposition_multicore.ml", line 5, characters 8-26:
5 | let k = Domain.DLS.new_key Random.State.make_self_init
            ^^^^^^^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-numerical/evolutionary_algorithm_multicore.ml", line 8, characters 13-31:
8 | let my_key = Domain.DLS.new_key Random.State.make_self_init
                 ^^^^^^^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-numerical/matrix_multiplication_tiling_multicore.ml", line 62, characters 36-48:
62 |   let domains = Array.map (fun c -> Domain.spawn (worker c)) channels in
                                         ^^^^^^^^^^^^
Error: Unbound module Domain
File "benchmarks/multicore-grammatrix/grammatrix_multicore.ml", line 136, characters 6-18:
136 |       Domain.spawn (fun _ -> worker samples res c))
            ^^^^^^^^^^^^
Error: Unbound module Domain
make: *** [Makefile:226: ocaml-versions/4.14.0.bench] Error 1

Reference: Build support for 4.14. https://github.com/ocaml-bench/sandmark/pull/387

Sudha247 commented 2 years ago

Thanks for the logs @shakthimaan! I see three separate issues here --

Lockfree: I presume lockfree doesn't building/install on 4.14 branch. This should be do-able given we have (mock) Atomic module 4.12 onward. I'll test this and update here.
Domains: The Domain module is not present in 4.14, hence some of the failures. We could have a mock-Domain similar to the mock-Atomics I mentioned above. Please let me know if it'd be useful to have.
C-API changes: Some C functions have changed in the 5.x branch. We could turn off the global-roots benchmark for 4.14, or replace it with the corresponding 4.14 version. These are micro-benchmarks not run in the nightly runs, I believe, so it might be okay to just turn them off.

shakthimaan commented 2 years ago

Yes, we can have mock Atomic and Domain modules with a README note on the same for 4.14. Sure, we can disable the global-roots benchmarks for 4.14. Please share a PR with the necessary changes, and I can test the same. Thanks!

ocaml-bench / sandmark

Multicore parallel benchmarks support for 4.14 #393