search

bcallaway11 / did

Difference in Differences with Multiple Periods, website: https://bcallaway11.github.io/did
287 stars 91 forks source link

Possible typo in the Wald pre-trend test for att_gt #167

Closed thameath closed 1 year ago

thameath commented 1 year ago

I noticed that the p-values for the reported pre-test of the parallel trends assumption in the att_gt() function were behaving a bit odd. It looks like there is a small typo in the calculation of the test statistic.

Currently it is coded as:

W <- n*t(preatt)%*%solve(preV)%*%preatt

Where n is the number of observations in the dataset, preatt is the matrix of ATT estimates in the pre-policy periods, and preV is the corresponding variance-covariance matrix. I'm pretty sure it shouldn't be scaling with the sample size, and removing the n multiplier produces results consistent with other implementation of the Wald test on multiple parameters:

W <- t(preatt)%*%solve(preV)%*%preatt
bcallaway11 commented 1 year ago

I don't think that this is correct -- you can check the way that V and preV are computed to see that the Wald statistic should include $n$. The test is for the joint test that all pre-treatment (pseudo) ATT(g,t)'s are equal to 0, so you may see some differences relative to, say, event study plots.

That said, if you have particular calculations that appear incorrect to you, I'd be happy to take a look.

Thanks, Brant

thameath commented 1 year ago

Oh, you are absolutely correct; I misread the way V was being calculated. Thanks for the clarification!