[Question] Why is ortho_init=False during hyperparameter searches?

[jkterry1]

It's my understanding that orthogonal initialization is basically universally slightly helpful and that this result has been verified in the literature, so it seems odd to disable it for hyperparameter tuning.

[araffin]

Hello,

basically universally slightly helpful

in practice, it mostly depends on the other hyperparameters (in my own experience, when doing hyperparameter optimization with ortho_init as an hyperparam, it was not an important one).

There are also other reasons:

• ortho_init does not only do orthogonal initialization, there is also some scaling for the value function last layer, setting it to False removes one trick (the less tricks the better, especially if it does not affect performance), see https://github.com/DLR-RM/stable-baselines3/blob/master/stable_baselines3/common/policies.py#L514
• ortho_init scaling is fixed (and I suspect that this likely not work well for wider/smaller architectures compared to the defaults)
• setting to ortho_init=False reduces the search space

[jkterry1]

Thanks a ton