search

Bluefog-Lib / bluefog

Distributed and decentralized training framework for PyTorch over graph
https://bluefog-lib.github.io/bluefog/
Apache License 2.0
291 stars 71 forks source link

Context for CUDA and NCCL #80

Open hanbinhu opened 3 years ago

hanbinhu commented 3 years ago

https://github.com/Bluefog-Lib/bluefog/blob/ec2a3fc4a3d3783d2ebcb22f4ebd4f961dd602e0/bluefog/common/mpi_context.h#L240-L240

hanbinhu commented 3 years ago

We should use a common context for MPI and NCCL controller for CUDA usage.