grf-labs / grf

Generalized Random Forests
https://grf-labs.github.io/grf/
GNU General Public License v3.0
938 stars 250 forks source link

How could I get the ATE/CATE.. using insrumental forest? #1348

Closed minhengw closed 10 months ago

minhengw commented 10 months ago

Hi, I am new to the IV forest. The follwoing part is moy code. It seems I could train the model but how do I get the results or the outputs of ATE/CATE of my IV forest? Thank you so much

X <- TwB[Xname] Y <- TwB[['stars']] W <- TwB[['special_occasion']] Z <- TwB[['business_revew_count']]

iv_forest <- instrumental_forest(X, # covariates Y, # outcome variable W, # binary treatment Z) # Instrumental Variable

Predict on out-of-bag training samples.

iv_pred <- predict(iv_forest)

erikcs commented 10 months ago

Hi @minhengw,

The CATEs are

iv_pred <- predict(iv_forest)$predictions

You can use the average_treatment_effect function for the ATE (thanks for pointing this out was missing from the doc example, we'll update it)

minhengw commented 10 months ago

Hi, thank you so much! Sorry for the last question:

if i use the saved grf object "iv_forest" to the function average_treatment_effect(): average_treatment_effect(iv_forest, target.sample = "all")

An error message generated: Error in get_scores.instrumental_forest(forest, subset = subset, debiasing.weights = debiasing.weights, : Average conditional local average treatment effect estimation only implemented for binary instruments.

Yes, my IV (Z) is NOT binary, but it seems that the algorithm accepts IV is binary or real. Is there anything I did incorrectly? Thx!

erikcs commented 10 months ago

average_treatment_effect only supports binary Z

minhengw commented 10 months ago

Got it, I may binarize the IV and try again. thx

minhengw commented 10 months ago

using binary IV generates na for estimate and se:

estimate std.err NaN NaN Warning message: In get_scores.instrumental_forest(forest, subset = subset, debiasing.weights = debiasing.weights, : Estimated instrument propensities take values between 0.006 and 1 and in particular get very close to 0 or 1. Poor overlap may hurt perfmance for average conditional local average treatment effect estimation.

If I further set the argumnet target.sample = "overlap":

Error in average_treatment_effect(iv_forest, target.sample = "overlap") : For any forest type other than causal_forest, the only implemented option is method=AIPW and target.sample=all

So, it seems IV forest (not conventional causal forest) only supports target.sample = all?

erikcs commented 10 months ago

Yes

erikcs commented 10 months ago

You could try estimating an ate over a subset with something like average_treatment_effect(forest, subset = forest$Z.hat < 0.95)

minhengw commented 10 months ago

Thank you so much! I tried some subsets of z.hat. Yes, some estimates are generated successfully, and the direction of effect was expected as well...

The only thing a bit weird is the magnitude of effect is EXTREMELY LARGE (i.e. the absoluete value > 400 or > 1000).