Change SAC library to use the implementation written by pranz24 - Githubissues

facebookresearch / mbrl-lib

Library for Model Based RL

MIT License

952 stars 154 forks source link

Change SAC library to use the implementation written by pranz24 #142

Closed luisenp closed 2 years ago

luisenp commented 2 years ago

Types of changes

Investigates the performance of MBPO when using this SAC implementation.

[x] Docs change / refactoring / dependency upgrade
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)

Motivation and Context / Related issue

As mentioned in #138, this library is reported to have better results than the one used previously.

How Has This Been Tested (if it applies)

Checklist

[x] The documentation is up-to-date with the changes I made.
[x] I have read the CONTRIBUTING document and completed the CLA (see CONTRIBUTING).
[x] All tests passed, and additional code has been covered with new tests.
[ ] Test on all domains to see if it works better than previous implementation.
[x] Tune hyperparameters for new implementation.
[ ] Update CHANGELOG

luisenp commented 2 years ago

From my first runs, the new SAC library improves results in cheetah and inverted pendulum, but not in the other domains yet. See plots below and compare with our results in the paper (still lower than orig. MBPO, but better than before).

I didn't individually tune for all domains, do that's the next thing I'll try.