Currently, the dataset class supports an X, Y object that is gonna be divided into train and validation data.
Having a X_test and Y_test attribute inside is needed to understand if we are overfitting to the train data. Usually, for the normal user this should be set to None (X_test, Y_test set to none, and the consumers of the dataset will ignore these attributes)
In the case, that the dataset has this, the consumers like SMAC and ensemble builder will report the performance through time automatically.
LMZimmer
commented
3 years ago
Currently, the dataset class supports an X, Y object that is gonna be divided into train and validation data.
Having a X_test and Y_test attribute inside is needed to understand if we are overfitting to the train data. Usually, for the normal user this should be set to None (X_test, Y_test set to none, and the consumers of the dataset will ignore these attributes)
In the case, that the dataset has this, the consumers like SMAC and ensemble builder will report the performance through time automatically.