Closed bangerth closed 3 years ago
Interestingly, the problem is not in MPI_Comm_split
: If I call
ierr = MPI_Comm_split(communicator,
/*color=*/0,
key,
&sub_comm);
I get as expected
Global rank 0 has key 1 and has sub-comm rank 1
Global rank 1 has key 0 and has sub-comm rank 0
Global rank 2 has key 1 and has sub-comm rank 2
Thanks for the report! A fix is coming in #8861 on master, and then we'll bring it back to the release branches.
Ha, how fun -- a one-character fix! Thanks for the quick patch, @jsquyres !
Thanks a lot, indeed!
Thank you for taking the time to submit an issue!
Background information
What version of Open MPI are you using? (e.g., v3.0.5, v4.0.2, git branch name and hash, etc.)
2.1.1 (packaged with Ubuntu 18.04)
Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)
Via Ubuntu 18.04
Please describe the system on which you are running
Details of the problem
MPI describes
MPI_Comm_split_type
as follows (copy pasting from the MPI 3.1 documentation):The emphasis is mine. I read this as follows: The rank within the new sub-communicator should be determined by the lexicographic ordering of the pairs
(key, rank-in-old-comm)
-- with first preference given to thekey
and second preference to the old rank. But this isn't happening, as shown in the following program where I want rank=1 inMPI_COMM_WORLD
to become rank=0 in the sub-communicator:When run with
mpirun -np 3 ./test
, I get this output:The final number should have been 1, 0, 2 instead.
Interestingly, though, while I can't seem to sort a rank to the front by choosing a small
key
, I can sort a rank to the end by choosing a large key. If I setthen I get this output:
One might suspect that the ordering of keys is turned around (largest keys get highest final ranks), but I really don't know whether that conjecture is true.
In any case, this came up in the context of https://github.com/dealii/dealii/pull/12022 .