CI for runtime testing (starter)

[naughtont3]

There are currently no specific tests for checking the runtime pieces, which becomes more important when updating the submodule pointers for openpmix/prrte. The intent of this ticket is to track the setup for initial set of tests to address this gap.

Plan

Add CI tests to Open MPI for checking runtime changes

Use MTT to capture the tests to run, that will then be run via the CI backend.

Additional context / notes from emails

• Note keep MTT config in 'ompi-tests' repo for easier setup/changes

• Number of nodes to use: 5 nodes (4ppn)

• Note, have ability to fine-tune number of nodes by adding comment in PR

  • https://github.com/open-mpi/ompi/wiki/PRJenkins#ibm-ci-fine-tuning

• OMPI CI container image may need to have MTT added

• Tests (round 1):

  • Trivial tests (sanity check)
  • IBM Tests: https://github.com/open-mpi/ompi-tests/tree/master/ibm

• Tests (future):

  • Map/bind/rank patterns to test (see also: contrib perl script, OMPI doc for common cases)
    • https://github.com/open-mpi/ompi-tests/tree/master/orte
  • Singletons (https://github.com/open-mpi/ompi/issues/11216#issue-1501370047)

TODO

• [ ] Basic runtime.ini for Trivial test (in ompi-tests repo)
• [ ] Update runtime.ini for IBM test suite
• [ ] Add build.sh/run.sh like prte/hello-world for easy CI integration
• [ ] Test/add to backend pipeline

[rhc54]

Remember that the ompi-tests repo has an orte directory in it that has at least some runtime-focused tests in it

[naughtont3]

added mtt starter bits (trivial and ibm) with update to PRRTE.py launcher https://github.com/naughtont3/mtt/tree/testing-runtime

need to look at starter version of build/run scripts

[naughtont3]

note: add build test with --enable-python-bindings to ensure bindings work/install

[naughtont3]

Update notes after sync-up w/ Josh today

• Rethinking the MTT PRRTE launcher path b/c mainly care about interactions with mpirun
• Also focus is on lots of options for map/bind, etc. that need checks on output to determine pass/fail
• Goal is to have CI checks for updating submodule pointer, etc.
• So mtt less critical for this kind of test running in CI context and therefore going to follow similar approach used for testing in pmix-tests repo.
• For now, not need any configure option changes, but in future may want to have the python binding option tested in OMPI or PRRTE CI
• Main tests of interest
  • keep tests in the ompi-tests-public repo for easy additions
  • helloworld w/ map/bind to core, node, and different options for report/display details via command-line args
  • MPI_Comm_Spawn (see also singleton tests in ompi-tests-public)
  • MPMD test

[jjhursey]

It took me a bit longer than expected, but I have the backend CI setup per our discussion.

The Open MPI Runtime Testing Harness is in the following PR:

• https://github.com/open-mpi/ompi-tests-public/pull/25

IBM's CI is setup to monitor the ompi-tests-public repo for any changes to the runtime directory and start the CI test defined in that directory in the virtual cluster. Look for the IBM CI (ppc64le, GNU, Runtime) status.

Once everything is working as expected in the ompi-tests-public repo, then I can enable it on the Open MPI repo for submodule updates.