search

ROCm / apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
BSD 3-Clause "New" or "Revised" License
17 stars 14 forks source link

support megatron seq_len > 4096 #135

Closed ramcherukuri closed 1 month ago

ramcherukuri commented 1 month ago

Ran L0 test and they passed.