MPI 4.1 - implement memory allocation kind info

open-mpi / ompi

Open MPI main development repository

https://www.open-mpi.org

Other

2.12k stars 856 forks source link

MPI 4.1 - implement memory allocation kind info #12189

Open hppritcha opened 8 months ago

hppritcha commented 8 months ago

This is a biggy and likely important to customers using Open MPI with GPU accelerators.

See https://github.com/mpi-forum/mpi-issues/issues/580 and entry 17 of section B.1.2 of the MPI 4.1 standard

edgargabriel commented 5 months ago

Details:

section 11.4.3 (Memory Allocation Info) of MPI 4.1 contains the description of the MPI memory allocation kinds concept
section 11.5 (Portable MPI Process Startup) contains the new argument to pass memory allocation kind to mpiexec
Side document (https://github.com/mpi-forum/mem-alloc) contains the name of the info values for the various accelerators (cuda, rocm, ze)

rhc54 commented 5 months ago

Just an FYI: you'll need to add an attribute definition to the PMIx standard if you want this to work with OMPI v5 and above - putting something in the MPIR side doc won't do anything for you, I'm afraid. I will add a PMIX_MEM_ALLOC_KIND definition to the PMIx library just so the support exists. The debugger guys look at the PMIx Standards doc, so somebody needs to get that added to it.

rhc54 commented 5 months ago

Also, could you perhaps add the list of the info values you mention (in case we need attributes for those too)? I can't leaf thru that side doc repo to find them, and they aren't shown anywhere obvious.

hppritcha commented 4 months ago

rhc54 commented 4 months ago

I have added the PMIx support and support for PMIx-based tools in PRRTE: see https://github.com/openpmix/prrte/pull/1973 and https://github.com/openpmix/openpmix/pull/3342. Still not sure if we need to worry about defining attributes for your identified values - is there such a need?