search

py-why / EconML

ALICE (Automated Learning and Intelligence for Causation and Economics) is a Microsoft Research project aimed at applying Artificial Intelligence concepts to economic decision making. One of its goals is to build a toolkit that combines state-of-the-art machine learning techniques with econometrics in order to bring automation to complex causal inference problems. To date, the ALICE Python SDK (econml) implements orthogonal machine learning algorithms such as the double machine learning work of Chernozhukov et al. This toolkit is designed to measure the causal effect of some treatment variable(s) t on an outcome variable y, controlling for a set of features x.
https://www.microsoft.com/en-us/research/project/alice/
Other
3.66k stars 692 forks source link

Effect Interval for DR Methods Not Possible When T0 is non-zero #813

Open holyguy737 opened 10 months ago

holyguy737 commented 10 months ago

I have 10 treatment levels 0,1,2,...,9. I wanted the confidence interval for treatment effects given a X value using the effect_interval function. I ran the code effect_interval(X=X[:1],T0=1,T1=2). The above function is working fine for DML algorithms but while using DR algorithms it is throwing the following error: AttributeError: Can only calculate inference of effects between a non-baseline treatment and the baseline treatment! In the documentation effect_interval is meant to work for different base treatment levels as the user wants. Another point is that, the above issue is not occurring while using the effect function. So, how can I get this issue solved?

vsyrgkanis commented 10 months ago

Its true. It was a software engineering technical reason and primarily because it required a lot more coding effort to implement intrrvals for these differences. Currently the linear drlearner fits separate ols models for each treatment (compared to baseline). So then you have CIs for each treatment compared to baseline.

but to get CIs for diffs between T2 and T1 you need to also calculate the covariance between the parameters that were estimated in the ols for T2 and the parameters that were estimated in the ols for T1. But these parameters live in different ols models and these ols models only calculate covariances for their own parameters.

To calculate the covariance between parameters that live in separate ols models, the structure of the code needs to change a lot.

There was also an alternative design that we had considered which however would require blowing up the dataset to be K times larger than the original dataset (where K is the number of treatments) and containing many zeros. This could have allowed you to estimate all parameters simultaneously in a single OLS for all treatments. To avoid this blow up in memory we didnt go with that design, as we thought there wouldn’t be that many uses.

if you really want intervals for T2-T1, then consider simply changing the baseline treatment to be T1 (e.g. by making T1 the lexicographically smallest treatment) when you fit the DRLearner and refit your model.

Then since T1 will be your baseline you will be able to get intervals for that difference.