Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
Critical (currently preventing usage)
Please provide a clear description of problem you would like to solve.
I am looking into the distributed manager. When creating a tree of communicators, it is possible to use a verbose flag to print the layouts. What it prints is something like this:
I think what would be good is to not print an empty list for leaf nodes. This is more or less cosmetic. However, what is very useful is to print a list of world ranks associated with every node: for example, if I have 8 ranks, and I have 2 model and 4 data parallel ranks, it would be good to see something like this:
This is very instructive to understand how ranks are placed and helps debugging. Especially when you are using say alltoall in one comm direction and all reductions in the other, you want to make sure that the alltoall ranks are placed closer together than the allreduce ranks. printing this topology helps to understand the comm layout better.
Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
Critical (currently preventing usage)
Please provide a clear description of problem you would like to solve.
I am looking into the distributed manager. When creating a tree of communicators, it is possible to use a verbose flag to print the layouts. What it prints is something like this:
Describe any alternatives you have considered
I think what would be good is to not print an empty list for leaf nodes. This is more or less cosmetic. However, what is very useful is to print a list of world ranks associated with every node: for example, if I have 8 ranks, and I have 2 model and 4 data parallel ranks, it would be good to see something like this:
world: [0, 1, ,2 ,3 ,4, 5, 6, 7] model = [ [0, 1, 2, 3], [4, 5, 6, 7] ] data = [ [0, 4], [1, 5], [2, 6], [3, 7] ]
This is very instructive to understand how ranks are placed and helps debugging. Especially when you are using say alltoall in one comm direction and all reductions in the other, you want to make sure that the alltoall ranks are placed closer together than the allreduce ranks. printing this topology helps to understand the comm layout better.