Optimize layer normalization for AMD GPUs

ROCm / apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

BSD 3-Clause "New" or "Revised" License

18 stars 14 forks source link

Closed hubertlu-tw closed 2 years ago

jithunnair-amd commented 2 years ago

@hubertlu-tw Is the unit test failure a flaky one, or related to this PR: test_loss_scale_decrease (test_checkpointing.TestCheckpointing)?

hubertlu-tw commented 2 years ago

@hubertlu-tw Is the unit test failure a flaky one, or related to this PR: test_loss_scale_decrease (test_checkpointing.TestCheckpointing)?

It's a flaky unit test failure from the upstream: https://github.com/NVIDIA/apex/issues/695.