search

uber / causalml

Uplift modeling and causal inference with machine learning algorithms
Other
5.02k stars 773 forks source link

Sensitivity Analysis for Multiple Treatments #299

Closed patriciowoodley closed 11 months ago

patriciowoodley commented 3 years ago

Hello,

First of all, thank you for such a wonderful library.

I am trying to use Uplift modeling in an A/B testing and I want to validate it.

I have read this notebook about sensitivity analysis but it only takes into account when you have one treatment

https://github.com/uber/causalml/blob/master/examples/sensitivity_example_with_synthetic_data.ipynb

Is it possible to do it for multiple treatments?

I have checked the docs and I am not sure how to pass the p_col parameter in the class Sensitivity because the propensity score is a dictionary in this case and the treatment column is not a binary vector.

ppstacy commented 3 years ago

Hi @patriciowoodley thanks for using CasualML!

Currently, the sensitivity analysis code itself only supports the single treatment use case. But I do think you can run this iteratively for different treatments which means you need to generate the data frame for every treatment and control following the below input. We will have df1 (treatment A + control), df2 (treatment B + control) etc., then use the different sensitivity analysis methods.

Let me know if this makes sense for you. Also if there are also some strong interests from other users, we can plan to extend this to multiple treatments.

image

patriciowoodley commented 3 years ago

Awesome. That was one alternative but I wanted to be sure that it makes sense.

I will give it a try

Thanks for your quick reply!