Test pmix/openmpi

[camillescott]

Some test commands:

• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --ns-dist 3:1 --fence "[db | 0:;1:]"
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --ns-dist 3:1 --fence "[0:]"
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --ns-dist 3:1 --fence "[b | 0:]"
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --ns-dist 3:1 --fence "[d | 0:]" --noise "[0:0,1]"
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --job-fence -c
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --test-publish
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --test-spawn
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --test-connect
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 5 --test-resolve-peers --ns-dist "1:2:2"
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --test-error
• [x] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --test-internal 4
• [ ] srun -t 20 --mem=1g -n4 -N4 /software/src/pmix/pmix-3.2.3/test/pmix_test -n 4 --ns-dist 3:1 --fence "[db | 0:0-2;1:3]"

[camillescott]

Test commands for PMIx v3.2.3: https://github.com/openpmix/openpmix/blob/ad8d1f3ac4d5ed6d0509d7c6129f20b59cc02d17/test/cmd_examples

[camillescott]

pmix_test -n 4 --ns-dist 3:1 --fence "[db | 0:0-2;1:3]" fails. This might be expected to fail, as 0:0-2;1:3 tries to use 3 ranks on the second fence, but it is only specified with 1 rank via --ns-dist 3:1. Need to investigate further.

[camillescott]

See also: https://slurm.schedmd.com/mpi_guide.html