search

pmodels / mpich

Official MPICH Repository
http://www.mpich.org
Other
538 stars 280 forks source link

Improvements to Virtual Topology Functions #1654

Open mpichbot opened 7 years ago

mpichbot commented 7 years ago

Originally by balaji on 2012-07-15 10:27:05 -0500

Improve the rank reordering support in the MPI topology functions to actually do some reordering.

EDIT: The list of functions that we should enhance:

MPI_Cart_map
MPI_Cart_create
MPI_Graph_map
MPI_Graph_create
MPI_Dist_graph_create
MPI_Dist_graph_create_adjacent
raffenet commented 5 years ago
  1. Use Scotch to map process ranks to the basic hardware topology (on-node vs. off-node).
  2. Incorporate additional hardware topology information to better refine the mapping.
    • on-node topology (hwloc)
    • off-node topology (netloc)
raffenet commented 5 years ago
  1. Use Scotch to map process ranks to the basic hardware topology (on-node vs. off-node).

  2. Incorporate additional hardware topology information to better refine the mapping.

    • on-node topology (hwloc)
    • off-node topology (netloc)

The current PR will address part 1. Part 2, as well as addressing the memory consumption of the communication matrix (O(p2)) can be done later, possibly as a student project.

hzhou commented 3 years ago

@raffenet Can you provide more descriptions/background on this ticket? I am not clear on what is supposed to do here.

raffenet commented 3 years ago

Today, when users ask MPI to remap the ranks in a communicator with topology (to optimized nearest neighbor communication), MPICH does nothing. This ticket is supposed to track work to implement this functionality in MPICH.

The reality is that many applications handle this scenario themselves, since they cannot rely on a portable solution from MPI. The PR linked used the Scotch library, but my preliminary investigation found little benefit to remapping for cartesian topologies, and we since moved on to other priorities.

hzhou commented 3 years ago

Thanks, @raffenet . Can you amend the description to list the functions that we are supposed to enhance? Is it just MPI_Cart_map?

raffenet commented 3 years ago

Thanks, @raffenet . Can you amend the description to list the functions that we are supposed to enhance? Is it just MPI_Cart_map?

MPI_Cart_map
MPI_Cart_create
MPI_Graph_map
MPI_Graph_create
MPI_Dist_graph_create
MPI_Dist_graph_create_adjacent