search

NVIDIA / apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
BSD 3-Clause "New" or "Revised" License
8.43k stars 1.4k forks source link

[Bug] Fix a bug in fused rope #1750

Closed yaox12 closed 1 year ago

yaox12 commented 1 year ago

Fix a bug in the fused rope backward calculation. cc @crcrpar