[Transformer][UCC] Fix async p2p ops

NVIDIA / apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

BSD 3-Clause "New" or "Revised" License

8.2k stars 1.36k forks source link

Closed Aidyn-A closed 1 year ago

Aidyn-A commented 1 year ago

This PR fixes the following failing test cases:

    test_bert_without_interleaving (test_bert_minimal.UccBertTest)
    test_gpt (test_gpt_minimal.UccGptTest)