rlworkgroup / garage

A toolkit for reproducible reinforcement learning research.

MIT License

1.85k stars 309 forks source link

DQN is not learning #1774

Open ahtsan opened 4 years ago

ahtsan commented 4 years ago

Would be great to be resolved along with #1773.

krzentner commented 4 years ago

Oh, this explains why refactoring the off-policy logging seemed to not be working for DQN.

ahtsan commented 4 years ago

For reference, 2020.06 version is still working (i.e. DQN is still learning) https://github.com/rlworkgroup/garage/blob/v2020.06.0/examples/tf/dqn_cartpole.py

ahtsan commented 4 years ago

1791 would probably help part of the issue.

maliesa96 commented 4 years ago

This seems to be caused by #1353, though I haven't yet figured out what LocalSampler and Frag Worker are missing. I'll backport this fix once I figure it out.