DeepLink-org / deeplink.framework

BSD 3-Clause "New" or "Revised" License
55 stars 28 forks source link

Fix unnecessary hccl dtype promotion #875

Closed zhaoguochun1995 closed 1 month ago

zhaoguochun1995 commented 2 months ago

Fix unnecessary memory type promotion in hccl: hccl supports int8, int16 but not uint8

在华为hccl适配中,避免bool 转成int32做通信,再int32转回bool。使用ne算子是为了bool类型求和时避免出现溢出,结果和cuda nccl通信结果对齐。 In Huawei hccl adaptation, avoid converting bool to int32 for communication and then converting int32 back to bool. The ne operator is used to avoid overflow when summing bool types, and the result is aligned with the cuda nccl communication result.

test case: dipu/tests/python/individual_scripts/test_rt_ddp.py