search

ja-thomas / autoxgboost

autoxgboost - Automatic tuning and fitting of xgboost
Other
122 stars 19 forks source link

Cross validation? #38

Open DanielKuehn87 opened 6 years ago

DanielKuehn87 commented 6 years ago

Are you currently using any form of CV? It kind of looks like you are using a holdout set for early stopping and then use this holdout set for the optimization.

ja-thomas commented 6 years ago

Yes you're correct. Since the optimization error is biased anyways we use the same. This not not 100% correct evaluation, but shouldn't make a huge difference (hopefully). This is also true for the threshold tuning.

We don't need an additional validation set with that approach. There is a branch with an additional validation set if you need it.