Closed mertalbaba closed 1 year ago
Performance of RL and imitation learning algorithms is very sensitive to hyperparameters and implementation details. Please try using the scripts we provide for training and tuned hyperparameters in https://github.com/HumanCompatibleAI/imitation/tree/master/benchmarking If those results don't line up with those in the paper https://arxiv.org/pdf/2211.11972.pdf then please do let us know.
The problem is that hyperparameters are only provided for some environments (no params. for Humanoid) and only provided for limited combinations (only with PPO). Do you plan to extend parameters for Humanoid and SAC/TRPO + GAIL/AIRL?
We don't currently have plans to extend them but we'd welcome PRs on this. You can use the scripts in https://github.com/HumanCompatibleAI/imitation/pull/675 to do that
Ok, thanks. I found hyper-parameters for GAIL-SAC in all Gym environments, will try to make a pull request for all this week.
@mertalbaba Hi! Thank you for raising this issue. Where can we find your params?
Bug description
Your adversarial model implementations, including GAIL and AIRL, does not work well in MuJoCo environments. Tested on Hopper, HalfCheetah and Humanoid, and both AIRL and GAIL failed to reach a meaningful score after 100k and 1 million steps of training.
Steps to reproduce
Environment
pip freeze --all
: