search

dmlc / xgboost

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
https://xgboost.readthedocs.io/en/stable/
Apache License 2.0
26.3k stars 8.73k forks source link

xgboost tweedie monotonicity violation #7830

Open ruthlessactuary opened 2 years ago

ruthlessactuary commented 2 years ago

Really appreciate your time looking over this and helping me find a solution or at least understand why I'm seeing what I'm seeing. I am fitting in python using 'objective': 'reg:tweedie' with the default greedy algorithm (not hist).

I am seeing some trees that seemingly violate my monotonicity. I have come up with a working theory that monotonicity is not necessarily enforcing the direction of predictions between left and right, but rather checking if the right prediction is less than or greater than 0. I don't know, and looking into the source code I see something about monotonicity being involved in setting allowable values. It's quite a lot to look through and I'm hoping for some guidance.

Attached is a picture. For intermediate nodes higher up from the leaf, I have used a weighted average of the leaf value & cover. I have continued this process up through the tree working backwards.

image

Thank you for any guidance!

trivialfis commented 2 years ago

Hi, thank you for opening the issue. Could you please share a reproducible example that we can run?

ruthlessactuary commented 2 years ago

Let me see if I can reproduce the issue without having to send the whole dataset. Thank you, I will reply once I do.