support collective operators - Githubissues

alibaba / BladeDISC

BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

Apache License 2.0

803 stars 160 forks source link

support collective operators #1289

Closed Yancey1989 closed 4 months ago

Yancey1989 commented 6 months ago

To optimize distributed training graph (DP, FSDP), DISC needs to support collective ops as a preliminary preparation

[x] support collective ops compilation and execution (all_reduce, all_gather, reduce_scatter) @Yancey1989
- 1275
- 1288
[x] support async collective ops execution @eedalong
- 1287