search

facebookresearch / ClassyVision

An end-to-end PyTorch framework for image and video classification
https://classyvision.ai
MIT License
1.59k stars 278 forks source link

Support FP16 grad communication in DDP #709

Closed mannatsingh closed 3 years ago

mannatsingh commented 3 years ago

Summary: DDP supports an fp16_compress_hook which compresses the grads to FP16, which can result in a significant speed up. Added support for this using the following config setting -

      "distributed": {
        "fp16_grad_compress": true
      },

Differential Revision: D26655070

facebook-github-bot commented 3 years ago

This pull request was exported from Phabricator. Differential Revision: D26655070

facebook-github-bot commented 3 years ago

This pull request was exported from Phabricator. Differential Revision: D26655070

facebook-github-bot commented 3 years ago

This pull request was exported from Phabricator. Differential Revision: D26655070

facebook-github-bot commented 3 years ago

This pull request was exported from Phabricator. Differential Revision: D26655070

facebook-github-bot commented 3 years ago

This pull request has been merged in facebookresearch/ClassyVision@668a8a140f516926bfc25f3935e9e328e7bc74c4.