To support experiments on whether our model selection strategy (maximal validation accuracy), we should provide a flag which enables model selection based on minimal validation loss instead.
The interface I would suggest is something like this: a flag --model_selection_mode with a default value accuracy and an alternative value loss. (We already have something similar in place for the reduce-on-plateau scheduling: see here).
To support experiments on whether our model selection strategy (maximal validation accuracy), we should provide a flag which enables model selection based on minimal validation loss instead.
The interface I would suggest is something like this: a flag
--model_selection_modewith a default valueaccuracyand an alternative valueloss. (We already have something similar in place for the reduce-on-plateau scheduling: see here).