Added 4-byte alignment on NCCL/RCCL (Cherry pick from upstream)

ROCm / DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

https://www.deepspeed.ai/

Apache License 2.0

5 stars 3 forks source link

Added 4-byte alignment on NCCL/RCCL (Cherry pick from upstream) #42

Closed amathews-amd closed 3 years ago

amathews-amd commented 3 years ago

Cherry pick from upstream 2c62843965f777ead99719202cd31f70fd3ad0a3 Upstream PR: https://github.com/microsoft/DeepSpeed/pull/1328

amathews-amd commented 3 years ago

@jithunnair-amd , please check.

jithunnair-amd commented 3 years ago

@rraminen Can you please publish a new rocm/deepspeed:rocm4.3.1_ubuntu18.04_py3.6_pytorch_1.9.0 image with this rccl_fix included for DeepSpeed? Please update on https://ontrack.amd.com/browse/MSRCHA-137 once you're done.

rraminen commented 3 years ago

@jithunnair-amd Okay