Allreduce composition with CLI support

microsoft / msccl-tools

Synthesizer for optimal collective communication algorithms

MIT License

98 stars 25 forks source link

Closed olsaarik closed 2 years ago

olsaarik commented 2 years ago

Implements the standard ReduceScatter+Allgather composition into an Allreduce. From the CLI this can be triggered with:

msccl compose allreduce reducescatter.json allgather.json