microsoft / msccl-tools

Synthesizer for optimal collective communication algorithms
MIT License
98 stars 25 forks source link

Questioning the optimality and generation strategy of Allreduce #54

Open nariaki3551 opened 1 year ago

nariaki3551 commented 1 year ago

Hi, I have questions about Allreduce synthesizing. In the SCCL paper, it is mentioned that the Allreduce algorithm is generated from the Allgather algorithm. I am curious about how much this approach guarantees the optimality of the Allreduce algorithm.

In particular, is there a possibility that obtaining Allreduce directly from a constraint problem as well as Allgather, could yield a more efficient or optimal algorithm than the current method?

Thanks.