Before we directly compared optimiser vs pretrain vs default parameters.
However, sometimes, the full optimiser doesn't have enough data to run but pretrain does.
Instead of ignoring these datapoints, we now default back to pretrain when optimiser fails, and default back to the default parameters when pretrain fails.
Before we directly compared optimiser vs pretrain vs default parameters. However, sometimes, the full optimiser doesn't have enough data to run but pretrain does. Instead of ignoring these datapoints, we now default back to pretrain when optimiser fails, and default back to the default parameters when pretrain fails.