search

pranz24 / pytorch-soft-actor-critic

PyTorch implementation of soft actor critic
MIT License
812 stars 181 forks source link

puzzles about action scaling #20

Closed wayunderfoot closed 4 years ago

wayunderfoot commented 4 years ago

Hi,thanks for your pytorch implementation of SAC,it's really readable. I have a question about action scaling when i read your code. https://github.com/pranz24/pytorch-soft-actor-critic/blob/cc42a1f31c0a5764548641ccbffe1561ba00622a/model.py#L100 What is its function if action_scale equals 1.0 and action_bias equals 0.And why can that make a big improment upon performence?

pranz24 commented 4 years ago

In practice, I don't think it ever made any difference in the performance. (at least on mujoco tasks). Stability-wise, I think, the performance should be more stable. (intuition :neutral_face:) You can go through this pull request to understand why we are doing action re-scaling: Fix bugs of action re-scaling

wayunderfoot commented 4 years ago

Thanks,i will check it out.