Update order of actor and critic

[hccz95]

It seems that you update critic before actor.

As far as I know, the actor_loss is calculated through critic network, so the backward of actor_loss will influence the grad of critic parameters.

Should we update actor first, and then update critic using both actor_loss and critic_loss?

[zhuzihan728]

In the original paper, they update critic first tho.