Reading https://github.com/vwxyzjn/cleanrl/blob/master/docs/rl-algorithms/sac.md and the code while implementing SAC with Jax (#300 ), several tweaks have been made compared to the original SAC implementation and I was wondering why and if the impact of each of those tweaks have been tested (apart from higher learning for the qf, which does yield slightly better results).
Reading https://github.com/vwxyzjn/cleanrl/blob/master/docs/rl-algorithms/sac.md and the code while implementing SAC with Jax (#300 ), several tweaks have been made compared to the original SAC implementation and I was wondering why and if the impact of each of those tweaks have been tested (apart from higher learning for the qf, which does yield slightly better results).
For a reference, I benchmarked my implementation here: https://wandb.ai/openrlbenchmark/cleanrl/reports/SAC-jax---VmlldzoyODM4MjU0 The only tweak I kept was the higher lr for the qf, the rest is the same as the original SAC implementation.
Related to https://github.com/vwxyzjn/cleanrl/pull/21
CC @dosssman
List of implementation details
Checklist
poetry install(see CleanRL's installation guideline.