Closed zzzxxxttt closed 5 years ago
replicate
is implemented using broadcast
. You can see that here: https://github.com/pytorch/pytorch/blob/v1.1.0/torch/nn/parallel/replicate.py#L97
replicate
is designed for network (nn.Module) replication. We just want to broadcast some tensors.
Get it, thank you very much!
Hi~ Thanks for providing this great implementation! I'm quite interested in the multi-gpu part which uses replicate for network weights and broadcast for the edge weights. I'm not familiar with parallel programming in pytorch, and I'm curious about the difference between broadcast and replicate, could you explain why we should use broadcast for edge weights?