Validation data generation, removal of loss function overrides

ppdebreuck / modnet

MODNet: a framework for machine learning materials properties

MIT License

80 stars 34 forks source link

Validation data generation, removal of loss function overrides #234

Open kaueltzen opened 3 weeks ago

kaueltzen commented 3 weeks ago

Resolves #228 .

Overview

Replaces train_test_split with explicit generation of validation data that is reproducible, shuffled and stratified (if classification tasks are present)
- as in #232 no explicit determination of classification targets for stratification, uses first target instead
Removes override of loss function for classification tasks

TODO

discuss: enable passing of loss function for model evaluation for classification tasks?

kaueltzen commented 5 days ago

Hi @ppdebreuck I have a question regarding this comment: https://github.com/ppdebreuck/modnet/issues/228#issuecomment-2437540148

Do you think it would make sense to enable the passing of a loss function for classification tasks in evaluate of MODNetModel? Or would you prefer keeping -ROC-AUC (e.g., because it's easier to interpret than cross entropy losses)?