Question re: Uplift Forest calibration

[zeromh]

Hi team, are you aware of any guarantees regarding Uplift Forest calibration, or familiar with any methods for calibrating the causalml Uplift Forest estimators?

The context here is that I fit an UpliftRandomForestClassifier on my data (control and two treatments, equal sample size for each, but very low target rates). The lift rankings of the resulting model are decent, but the lift calibration is completely wrong. I went and checked Rzepakowski's paper (Decision trees for uplift modeling with single and multiple treatments) on these estimators, and I see that the authors only checked estimator rankings/profits, but not calibration.

My use case is to actually use the outputs from the estimator as real probabilities, so I need good calibration. Are you aware of any work describing Uplift Forest calibration in general? Or do you know any calibration classes that work natively on your estimators?

[Ramlinbird]

Hello, I wonder if you found any solution, because I met with similar problem, the predict uplift is not match with the actual uplift (nearly predict all positive, but some has negative effect). Here's my original question, https://github.com/uber/causalml/discussions/655

[zeromh]

I did not find a solution for the UpliftRandomForestClassifier. However, it looks like your question is not necessarily about calibration. (A qini curve is for checking uplift rankings, not calibration.) I'll respond there.

[zeromh]

@vincewu51 Saying "no, I'm not familiar with any methods for calibrating the causalml Uplift Forest estimators" would have been kinder than closing the issue and marking it as "completed" without even responding.

[vincewu51]

Hi @zeromh Thanks for raising this. I was trying to clean up the whole backlog yesterday, mistakenly thought it is solved. I've re-opened it and mark it as "question".

[zeromh]

Thanks!