Pytorch SAC alpha sign

[jose-alatorre-harvard]

Hi,

In line 241 of sac.py loss_pi = (alpha * logp_pi - q_pi).mean()

shouldnt be ? loss_pi = (q_pi-alpha * logp_pi ).mean()

I believe the signs are wrong.

[Harimus]

optimizing (taking learning steps) in pytorch are done using gradient decent on the loss function, while in policy gradients methods, we use gradient ascent to improve the actor. Switching the sign here is to make the gradient decent have the gradients ascent effect. The compute_loss(obs, act, weight) in Spinningup Intro3 uses the same trick.