A FasterRMSNorm implementation (based on FasterLayerNorm)

NVIDIA / apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

BSD 3-Clause "New" or "Revised" License

8.2k stars 1.36k forks source link

Open Njuapp opened 1 year ago

Njuapp commented 1 year ago

A FasterRMSNorm implementation (similar to FasterLayerNorm).