Training using AsyncReplay gets stuck after arounf 50k steps.

ShangtongZhang / DeepRL

Modularized Implementation of Deep RL Algorithms in PyTorch

MIT License

3.18k stars 682 forks source link

Training using AsyncReplay gets stuck after arounf 50k steps. #78

Closed ayooshkathuria closed 4 years ago

ayooshkathuria commented 4 years ago

Whenever I use AysncReplay for replay_fn, my training gets stuck at around 40-50k steps. I am able to get results using Replay and async_actor = False, but any idea why the former is causing the training to hang?

Also, will using AyncReplay or Replay cause difference in performace?

ShangtongZhang commented 4 years ago

50K steps is when training starts. AsyncReplay is much faster than Replay. Did you use the Dockerfile provided?