add ROCm devices support

bigcode-project / Megatron-LM

Ongoing research training transformer models at scale

Other

371 stars 48 forks source link

Closed mayank31398 closed 1 year ago

mayank31398 commented 1 year ago

Allows the codebase to run on AMD Instinct GPUs

loubnabnl commented 1 year ago

Yes I managed to launch the training on AMD GPUs (also tested on A100 it works fine)