Open hccz95 opened 1 year ago
It seems that you update critic before actor.
As far as I know, the actor_loss is calculated through critic network, so the backward of actor_loss will influence the grad of critic parameters.
Should we update actor first, and then update critic using both actor_loss and critic_loss?
In the original paper, they update critic first tho.
It seems that you update critic before actor.
As far as I know, the actor_loss is calculated through critic network, so the backward of actor_loss will influence the grad of critic parameters.
Should we update actor first, and then update critic using both actor_loss and critic_loss?