[CAY-1259] Efficient push using multi-update

snuspl / cruise

Cruise: A Distributed Machine Learning Framework with Automatic System Configuration

Apache License 2.0

26 stars 2 forks source link

[CAY-1259] Efficient push using multi-update #1260

Closed wynot12 closed 6 years ago

wynot12 commented 6 years ago

Resolves #1259

This PR changes ModelAccessor to use multi-update API in order to reduce the overhead.

wynot12 commented 6 years ago

If processing rate of push requests are slower than generation of requests, the sender queue becomes full and the iteration time keeps growing to wait for putting request into queue. In the below graph, we can see that push-improved version (orange) handles the workload (NMF) well, which has been a problem in the original version (blue).