search

py-why / dowhy

DoWhy is a Python library for causal inference that supports explicit modeling and testing of causal assumptions. DoWhy is based on a unified language for causal inference, combining causal graphical models and potential outcomes frameworks.
https://www.pywhy.org/dowhy
MIT License
6.9k stars 918 forks source link

Conditional Average Treatment Effects (CATE) with DoWhy and EconML #334

Open ruiiliuu opened 2 years ago

ruiiliuu commented 2 years ago

Hi, I'm new in the field of Causality and I would like to use your package for my current project. This is a really great toolbox for causal inference. But I have several questions when checking case studies.

  1. https://microsoft.github.io/dowhy/example_notebooks/dowhy-conditional-treatment-effects.html (Conditional Average Treatment Effects (CATE) with DoWhy and EconML) In this case, the treatment Vo is continuous data, but in the estimation step, why do designers define the control value as 0 and the treatment value as 1? I am also wondering if I have both treatment and control groups, where I could define them? And if the sample number in two groups is imbalanced, will it influence my causal effect?

dml_estimate = model.estimate_effect(identified_estimand, method_name="backdoor.econml.dml.DML", control_value = 0, treatment_value = 1, target_units = lambda df: df["X0"]>1, # condition used for CATE confidence_intervals=False, method_params={"init_params":{'model_y':GradientBoostingRegressor(), 'model_t': GradientBoostingRegressor(), "model_final":LassoCV(fit_intercept=False), 'featurizer':PolynomialFeatures(degree=1, include_bias=False)}, "fit_params":{}}) print(dml_estimate)

  1. As discussed, different estimation methods could get quite different results. Then if I do not have true ATE to compare. How could I believe my estimation method?

Thanks in advance for the explanation.

amit-sharma commented 2 years ago
  1. For your first question, using treatment=1 and control=0 is simply a convention for continuous treatment variables. You can set any value. That depends on your application and which value of treatment will actually be administered. Sample imbalance can definitely influence the quality of your estimate, but is not a big problem unless it is extreme. You need some diversity in your treatment.
  2. Without true ATE, you can use the refutation methods from DoWhy to test the validity of your estimate. Check the getting started notebook. These refutation tests can be used to filter out bad estimates, but still do not validate the analysis fully. It is impossible to do that without comparing to a randomized experiment or true ATE.
EgorKraevTransferwise commented 2 years ago

I am using the ERUPT metric on a holdout dataset to compare different models, seems to make sense to me, namely out-of-sample comparison of treatment assignment policies conditioned on the CATE estimated by the different models.

FrancyJGLisboa commented 1 year ago

Hi @amit-sharma. Thanks for the answer to @ruiiliuu as those were one of my questions as well. I'd suggest making it clear in the example notebook. In my project, I have Y (continuous) and T (continuous) with a bunch of confounders (X,W). In my case, I want to answer a question. What would happen to T if we move from T to T*0.95 in the next year? What I want is to have an estimate of this effect for each subject (N=165). How would I set it?