Multi-node tests on Borealis

[mateuszpn]

Suspended, passed to @lslusarczyk The status:

• qsub + mpirun working, running the multinode benchmarks (single node temporary disabled in the branch)
• plotter generating only part of figures
• not proper control (--nodes param not properly interpreted, --different_devices switches to multinode instead) Current state of code: 1) https://github.com/mateuszpn/distributed-ranges/tree/multinode 2) https://github.com/intel-sandbox/libraries.runtimes.hpc.dds.dr-ci/tree/dev/multi-node

For quick start:

• clone the 2) repo into ci directory
• cd ci
• run the script:

  rm -rf dr
  git clone https://github.com/mateuszpn/distributed-ranges.git dr && cd dr && git switch multinode && cd ..
  /usr/bin/python3.10 -m venv venv
  source venv/bin/activate
  pip install dr/src-python/drbench
  CXX=icpx CC=icx MKL_DIR=${MKLROOT}/lib/cmake/mkl TBB_DIR=${TBBROOT}/lib/cmake/tbb cmake -B dr/build -DENABLE_SYCL=on dr
  scripts/borealis-reserve-nodes.sh 8
  scripts/borealis-benchmark-multi.sh