[FEATURE REQUEST] Modularized Distributed Optimizer

MagmaDNN / magmadnn

MagmaDNN: a simple deep learning framework in c++

MIT License

45 stars 7 forks source link

[FEATURE REQUEST] Modularized Distributed Optimizer #8

Closed Dando18 closed 4 years ago

Dando18 commented 5 years ago

Add a distributed SGD optimizer to streamline the process of distributed training. This removes the necessity to write custom MPI/NCCL programs for distributed training.

[x] implement MPI distributed optimizer
[ ] implement NCCL distributed optimizer
[ ] profile and compare optimizers
[x] test optimizers

Dando18 commented 4 years ago

Closed in 1f86e36c693a5329ef7700f694b33c9ff3873bb9