Optimize actor before the critic

rail-berkeley / rlkit

Collection of reinforcement learning algorithms

MIT License

2.49k stars 553 forks source link

Optimize actor before the critic #109

Closed Xingyu-Lin closed 4 years ago

Xingyu-Lin commented 4 years ago

The actor should be optimized first. Otherwise, the weights of the cirtic would have been changed when back propagating the gradients for the actor. The latter will create an error in pytorch 1.5.0

ksluck commented 4 years ago

Came here to propose the same pull request because I get the same error with the newest version of pytorch as Xingyu-Lin. Would be nice to have this patched in soon!