search

elephaint / pgbm

Probabilistic Gradient Boosting Machines
Apache License 2.0
138 stars 20 forks source link

Question: on the computation of mean for point/probabilistic estimates #9

Closed ivan-marroquin closed 2 years ago

ivan-marroquin commented 2 years ago

Hi,

Thanks for making available such great package!

This is not a bug but rather a simple question, I noticed that an "initial estimate" is added to the mean. Could you explain what is the reason? And also, what is the advantage/difference with respect to substract or not to include the "initial estimate"?

Best regards,

Ivan

ivan-marroquin commented 2 years ago

I forgot to mention that I referred to lines codes 400 and 447 in https://github.com/elephaint/pgbm/blob/main/src/pgbm_dist/pgbm_dist.py

elephaint commented 2 years ago

Hi,

Thanks for your kind words! The initial estimate is the mean over all the samples in the training set. This is a common initialization for GBMs in regression problems, because it speeds up the training. Because we start training with this initial estimate, and subsequently build a new tree based on the gradients (thus, the error of this initial estimate), we also need to add this initial estimate in our final prediction to the estimated mean. (I am working on including an initial variance too, but left that out for now).

Hope this explains it,

Olivier

ivan-marroquin commented 2 years ago

Hi Olivier,

Thanks for the explanation!

Ivan