Explain how predict_proba is computed from (Hist)GradientBoostingClassifier

scikit-learn / scikit-learn

scikit-learn: machine learning in Python

https://scikit-learn.org

BSD 3-Clause "New" or "Revised" License

58.76k stars 25.12k forks source link

Explain how predict_proba is computed from (Hist)GradientBoostingClassifier #29163

Open baggiponte opened 4 weeks ago

baggiponte commented 4 weeks ago

Describe the issue linked to the documentation

I think users would find it useful to see how class probabilities are computed for gradient boosting classifiers 😊 Tried to look into the source but could not understand it. Would be up to taking care of this, but I'd need a reference. Thank you!

Suggest a potential alternative/fix

No response

adrinjalali commented 4 weeks ago

@lorentzenchr can probably give a better intuition here.

lorentzenchr commented 4 weeks ago

See user guide:

For binary classification: $\mathrm{expit}(\sum_i tree_i(X))$ and $i=1...$ number of trees/iterations
For multiclass class $k$: $\mathrm{softmax}(trees(X)) = \frac{\exp(\sum_i tree_i^k(X))}{\sum_c \exp(\sum_i tree_i^c(X))}$.

baggiponte commented 4 weeks ago

Will use the refs and open a PR later (hopefully today). Will tag @lorentzenchr too for review 😊