Closed khumairraj closed 3 years ago
" In both the above cases (data with folds or not with folds), the grid point will first be found followed by using this best grid point configuration to train on the train+valid set. Thereafter, the final testing scores will be computed."
This should be optional, otherwise we are breaking the behavior that we used for the open tasks.
We might want an alternate mode where, for each train/val/test set, we do a full grid search. (Not one grid search and keep the grid point for the other folds.) This might be useful for small datasets.
Inherited by #315 . Closing this
This will
n
sets ofdata_splits
. Result for testing of each fold will be provided along with the aggregated result.folds setup
, will only be found with the firstdata_splits
. This will correspond to testing on the first fold(fold0), validation on second fold(fold1) and training on all other folds(fold2..). ( Sort the folds before deciding order(first or second) of the fold.test, train and valid
splits defined, best grid point will be found using the whole data.train+valid
set. Thereafter, the final testing scores will be computed.