Bug fix for SAC-discrete.

toshikwa commented 4 years ago

Hi, thank you for your great work :)

I fixed some bugs related to #54 and #56. I tested it in CartPole.py and saw the training converged more stably.

Changes

[x] fix min_qf_next_target to properly calculate expectations over policy.
[x] similarly fix policy_loss.
[x] fix max_probability_action to properly get argmax over each sample, not entire batch. (It doesn't actually affect the algorithm, though.)
[x] fix device for Replay_Memory to train SAC-Discrete on CPU.
[x] fix errors due to the update of PyTorch (now torch>=1.4.0 works!!).

Thanks :)

p-christ commented 4 years ago

thanks a lot

toshikwa commented 4 years ago

Hi @p-christ

Thank you for merging. Could you close relevant issues(54, 56)??

Also, I found that this repo got older and there were a few bugs. Can I fix bugs and clean up codes??

Thanks :)

p-christ commented 4 years ago

Sure will close them. Yes pls do clean up the code and do a new pull request!

On Sat, Sep 19, 2020 at 21:58, Toshiki Watanabe notifications@github.com wrote:

Hi @p-christ https://github.com/p-christ

Thank you for merging. Could you close relevant issues(54, 56)??

Also, I found that this repo got older and there were a few bugs. Can I fix bugs and clean up codes??

Thanks :)

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/p-christ/Deep-Reinforcement-Learning-Algorithms-with-PyTorch/pull/60#issuecomment-695355392, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGJAGA6AU2JDIJ3WD7WFR6TSGULPJANCNFSM4RRFGROA .

p-christ / Deep-Reinforcement-Learning-Algorithms-with-PyTorch

Bug fix for SAC-discrete. #60