Open pearce8 opened 1 month ago
@rfhaque Please take a look at how this PR gets around the questions, but we need a longer-term plan.
GTL: GTL is GPU Transport Layer, an HPE library required for CUDA-aware MPI.
[ ] Currently using GTL everywhere. Is this correct? [ ] We are not using CrayMPI on lassen - should we use the GTL or non-GTL version? [ ] What version should we use on Pascal (also no CrayMPI) [ ] Rocm experiments: do we need GTL? Make them match the Cuda experiments
@rfhaque Please take a look at how this PR gets around the questions, but we need a longer-term plan.
GTL: GTL is GPU Transport Layer, an HPE library required for CUDA-aware MPI [ ] Currently using GTL everywhere. Is this correct? [ ]
We are using gtl on tioga. I think we should use whichever is the default MPI version on each platform (and remove all references to gtl from ramble.yaml
We are not using CrayMPI on lassen - should we use the GTL or non-GTL version? [ ]
We use spectrum-mpi (non-gtl) which is the default mpi on lassen
What version should we use on Pascal (also no CrayMPI) [ ]
We use mvapich2-2.3.7-gcc-11.2.1 on Pascal, that is not gtl enabled
Rocm experiments: do we need GTL? Make them match the Cuda experiments
We do not need gtl in the rocm experiments. We can use the default mode provided by the underlying MPI implementation on the platform
Types of experiments
What is working: