search

uber / causalml

Uplift modeling and causal inference with machine learning algorithms
Other
4.9k stars 762 forks source link

Application to observational data ? #204

Closed Arminius4 closed 3 years ago

Arminius4 commented 4 years ago

Dear team, the Whitepaper to causalml states that

the current implementation of the uplift modeling is encouraged to be applied to the data from the randomized experiment. Applications to observational data where the treatment is not assigned randomly should take extra caution. In nonrandomized experiment, there is often a selection bias in the treatment assignment (a.k.a. confounding effect).

We plan to apply causal inference to observational medical data, where we already know that our data is confounded by clinical guidelines for selecting treatments (which is a motivation to apply causal inference techniques). Could you please explain what is missing in the current implementation for the application of uplift modeling to observational data and what it means to take extra caution?

t-tte commented 4 years ago

Thanks for the great question. There's nothing missing from the package when it comes to observational data. It's just that the uplift modelling methods we've implemented assume what's often called conditional ignorability, which is broadly speaking the idea that your feature set X contains the relevant confounders, or more specifically blocks any back-door paths between the candidate cause and the outcome of interest.

In randomised experiments, the ignorability assumption is satisfied by design. In observational studies, we cannot tell for sure whether it is satisfied or not. Instead, we usually need to rely on subject matter expertise to determine what the relevant back-door paths could be, and what variables we need to "control for" to be able to estimate the treatment effect that we are interested in. When we talk about "extra caution" in the context of observational studies, we refer to the need for substantive subject matter expertise.

There is another interesting problem when it comes to heterogeneous treatment effect estimation with observational data, which is that there is very little systematic research on how the different algorithms break down in the presence of confounders, both measured and unmeasured. Consequently, we are currently not able to recommend a particular algorithm for observational data, or tell you how badly your estimates are going to be wrong under different confounding scenarios. I'd expect some researchers to be working on this problem given that there seems to be a lot of demand for heterogeneous treatment effect estimation in observational studies.