The default in scikit-learn's SearchCV is to use the learning algorithm's score function. E.g. SVM will use accuracy. This could be a problem if the data is imbalanced. It would be better to use balanced_accuracy.
To prevent this, ideally we should warn the user if:
1) Hyperparemeter tuning is happening
2) Scoring is implicit
Additionally, this should include an example
How do you imagine this integrated in julearn?
In the checks, if a SearchCV is used and the scoring is None
Do you have a sample code that implements this outside of julearn?
Which feature do you want to include?
The default in scikit-learn's SearchCV is to use the learning algorithm's
scorefunction. E.g. SVM will useaccuracy. This could be a problem if the data is imbalanced. It would be better to usebalanced_accuracy.To prevent this, ideally we should warn the user if: 1) Hyperparemeter tuning is happening 2) Scoring is implicit
Additionally, this should include an example
How do you imagine this integrated in julearn?
In the checks, if a SearchCV is used and the scoring is
NoneDo you have a sample code that implements this outside of julearn?
No response
Anything else to say?
No response