facebookresearch / fairscale

PyTorch extensions for high performance and large scale training.
Other
3.18k stars 280 forks source link

Changed to only run reshard hook if all gradients computed #1166

Closed awgu closed 7 months ago

awgu commented 8 months ago

What does this PR do?

Fixes # (issue).

Before submitting

PR review

Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.