hearbenchmark / hear-eval-kit

Evaluation kit for the HEAR Benchmark
https://hearbenchmark.com
Apache License 2.0
56 stars 17 forks source link

[WIP] Add Training for Data with Folds; Do full training on test and valid before final testing #310

Closed khumairraj closed 3 years ago

khumairraj commented 3 years ago

This will

turian commented 3 years ago

" In both the above cases (data with folds or not with folds), the grid point will first be found followed by using this best grid point configuration to train on the train+valid set. Thereafter, the final testing scores will be computed."

This should be optional, otherwise we are breaking the behavior that we used for the open tasks.

turian commented 3 years ago

We might want an alternate mode where, for each train/val/test set, we do a full grid search. (Not one grid search and keep the grid point for the other folds.) This might be useful for small datasets.

khumairraj commented 3 years ago

Inherited by #315 . Closing this