search

facebookresearch / mbrl-lib

Library for Model Based RL
MIT License
950 stars 154 forks source link

hyperparameters optimization #166

Open ss555 opened 2 years ago

ss555 commented 2 years ago

🚀 Feature Request

I would like to optimize the hyperparameters on a custom environment for PE-TS and other algorithms.

Motivation

How did you find the optimal hyperparameters for the algorithms? for example PE-TS cartpole

Pitch

PE-TS example I did the grid search for 4 parameters: horizon_size, alpha, number of hidden layers, hidden layer dimension.

problems: what parameters are more crutial to optimize.

Do you have bayesian optimisation script for hyperparamters

Describe alternatives you've considered I can make a pull request for the PE-TS grid search or/and bayesian optmization with optuna library.

luisenp commented 2 years ago

Hi @ss555, thanks for the interest and the offer to open a PR. Just wanted to quickly mention that it's currently possible to do some hyperparameter tuning of our examples by using hydra's with the Nevergrad plugin. The parameters on our defaults configs were indeed found that way using a small random search, but Nevergrad supports much more sophisticated algorithms.

On the other hand, AFAIK, Nevergrad doesn't support Bayesian optimization, so adding this could be interesting.