Chapter 7: 03_dqn_double.py

PacktPublishing / Deep-Reinforcement-Learning-Hands-On

Hands-on Deep Reinforcement Learning, published by Packt

MIT License

2.83k stars 1.28k forks source link

Chapter 7: 03_dqn_double.py #29

Closed JTatts closed 5 years ago

JTatts commented 5 years ago

Hi,

I've been playing with your double dqn implementation and have found that unfortunately the model does not seem converge.

To fix this I added a '.detach()' to line 30 because I was worried that backpropping the net for the action selection could be the cause of the issues. This seems to work but I have to admit that I'm confused as to why. I would have thought that the detach in line 36 would already block the gradients.

Do you have any idea what might be going on?

Cheers, Jamie

hemanthsavasere commented 5 years ago

Hi @JTatts, I am facing the same problem, please let me know if this is resolved.

Thanks

Shmuma commented 5 years ago

Hi!

It shouldn't be the case, as detach is called on a later expression (line 36), which effectively detaches full subbranch from the graph. It might be unlucky seed situation, have you tried to run it several times?