search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.78k stars 6.79k forks source link

Does mxnet support bounded-delay consistency model now? #8464

Open chongyang-xu opened 7 years ago

chongyang-xu commented 7 years ago

Description

I read the paper of MXNet

MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems

in part 2.3,it says "Currently, we support the sequential and eventual consistency." however ,th kv-store implemetion in parameter server support bounded-delay consistency model.

I want to know whether mxnet support bounded-delay consistency model now? And what method the distributred traning process deploy? BSP, SSP or other ?

formath commented 7 years ago

distributed mxnet support BSP and ASP. SSP is rarely used in the real-world now.

eric-haibin-lin commented 7 years ago

bounded-delay has not been added to MXNet yet. Feel free to contribute :)