Compare the performance of different distributed training + parallel training algorithms

sxjscience commented 8 years ago

We can change the update logic here https://github.com/peterzcc/dist_train/blob/arena/dqn_dist_demo.py#L277 to use KV Store.

sxjscience commented 8 years ago

Downpour Adagrad ,EA-Adagrad and MEASGD has been added in the new version. To test these algorithms, we need to save parameters periodically from the server.

We need to make a fair comparison between different distributed reinforcement learning algorithms. In fact, we can plot the training time + reward for some selected Atari2600 games.

Selected Games (We may add/delete some games)
Breakout
Q*bert
Seaquest
Space Invaders

Single Machine Tests (We have done this part?)

[ ] RMSProp
[ ] Adagrad

Downpour Adagrad (Communication Interval = 1, 4)

[ ] Two Machines
[ ] Three Machines
[ ] Four Machines

EA-Adagrad (Communication Interval = 4, 16, 64)

[ ] Two Machines
[ ] Three Machines
[ ] Four Machines

MEASGD (Communication Interval = 4, 16, 64)

[ ] Four Machines

@peterzcc

sxjscience commented 8 years ago

Also, I find this paper http://arxiv.org/pdf/1602.02672v1.pdf, which is related to our current distributed environment.

peterzcc / Arena

Compare the performance of different distributed training + parallel training algorithms #2