search

chaisemartinPackages / did_multiplegt_dyn

|| Stata | R || Estimation of event-study Difference-in-Difference (DID) estimators in designs with multiple groups and periods, and with a potentially non-binary treatment that may increase or decrease multiple times.
23 stars 6 forks source link

SE inconsistency in did_multiplegt_dyn STATA #53

Open fcabrerahz opened 2 weeks ago

fcabrerahz commented 2 weeks ago

Hello all,

In august 2023 I used the did_multiplegt_dyn command in Stata to evaluate an unconditional scholarship for individuals in primary schools in Mexico. Recently, one government official tried to replicate my results with an updated version of the did_multiplegt_dyn command, and now the standard errors are very different, now are very big (see images below).

The setup is the next one: from 2018 to 2023, I have schools treated with a scholarship at the student level and others non-treated. I used continuous and binary definitions of the treatment. I have a staggered implementation. In the binary treatment, the one I show below, my definition is “once treated, always treated”, then we do not have schools switching in and out, only switching in.

I have read a 2022 version of the paper about intertemporal treatment effects and the most recent version from January 2014. I cannot see differences in the section “inference”.

This image shows my “original” estimations of the effect of the scholarship on dropout rates at the school level:

image

I have some problems of pretrends, but that is not my concern. My real problem now is that with the exact same data and specifications ---but with a newer version of the command in STATA, I get this:

image

Furthermore, replicating this with the newest version of the command in R, I get something more similar to what I used to obtain with the “old” did_multiplegt_dyn. For example:

image

My specification in Stata is this:

did_multiplegt_dyn abandonoi school_id year treat_med, /// effects(3) placebo(2) effects_equal controls($controls) trends_nonparam(pan_marg) /// weight(alumni) cluster(munid) ///

Taking out the non_parametric trends does not change much the story. Also including the less_conservative_se option does not change the situation. What does change my SE, and actually make them more similar to my previous ones, is getting rid of the cluster option (I cluster at the municipality level).

I would really appreciate any help or information on updates on how the standard errors are now computed in STATA. Many thanks in advance.

Best,

Dr. Francisco Cabrera CIDE, Mexico City.

fcabrerahz commented 2 weeks ago

I have now seen that it is the clustered SE computation that causes the divergence. I can more or less replicate my old and the R-software standard errors if I use the cluster() option at the group level and not at the municipality. What specifically changed in the Stata command when clustering SE?