Closed nzw0301 closed 4 months ago
@not522 @eukaryo Could you review this PR?
Sorry for my late response.
How about using sklearn.metrics.roc_auc_score(valid_y, preds)
? It seems like a more usual way of calculating AUC. (I'm not familiar with LightGBM, so please correct me if I'm wrong.)
Thank you for your suggestion! You're absolutely right and my comment:
Compute AUC score using sklearn: We cannot calculate the AUC score because gbm does not have predict_prob method https://github.com/dmlc/xgboost/issues/4583, which is necessary to compute the score
was wrong. I'll update the file by following your suggestion.
Motivation
Resolve https://github.com/optuna/optuna-examples/issues/251.
Description of the changes
Use pre-computed booster's validation auc score rather than computing inconsistent accuracy.
Alternative ways I considered were not working by the following reasons:
gbm
does not havepredict_prob
method https://github.com/dmlc/xgboost/issues/4583, which is necessary to compute the score