Closed wangraying closed 2 years ago
refer to: https://pytorch.org/docs/stable/_modules/torch/nn/parallel/distributed.html#DistributedDataParallel.no_sync
it is now supported by https://github.com/BaguaSys/bagua/pull/312
refer to: https://pytorch.org/docs/stable/_modules/torch/nn/parallel/distributed.html#DistributedDataParallel.no_sync