search

bcallaway11 / did

Difference in Differences with Multiple Periods, website: https://bcallaway11.github.io/did
298 stars 95 forks source link

P-value for pre-test of parallel trends assumption = 0, dynamic aggregation of ATTs not below p-val threshold pre-0 event time #186

Open emmiobara opened 10 months ago

emmiobara commented 10 months ago

Hello,

We're using the did package and the output of the p-value of the Wald statistic for pre-testing the parallel trends assumption = 0 in many of our analyses. However, the dynamic aggregation of the ATTs does not show any of the event times pre-zero as statistically significant. The summary of the group-time ATEs also do not show any of the pre-treatment periods as statistically significant. In addition, all of our plots show the confidence intervals of the pre-zero periods crossing 0. We have checked the coding of the group and time variables and they seem to fit the required data structure.

Any ideas as to why the p-value for the pre-test of parallel trends is 0?

Thanks!

bcallaway11 commented 10 months ago

Hi @emmiobara, will you post here or email me a screenshot of the estimated ATT(g,t)'s that you are getting.

Thanks, Brant

emmiobara commented 10 months ago

Hi @bcallaway11 ,

I've sent you a screenshot of the output via email.

Emmi

AnneSimon233 commented 7 months ago

Hi to both of you,

were you able to sort out this issue? I'm experiencing the same issue and would like to be able to get the actual p-values for pre-trends...

Thanks a lot in advance!

Best, Anne

emmiobara commented 7 months ago

Hi Anne,

I was not able to figure out this issue. Unfortunate, but I'm weirdly glad to see that this isn't just an issue with our data.

Best, Emmi

lgoetschius-hilltop commented 4 months ago

Hello,

I wanted to follow-up on this issue as well. My team is working on a program evaluation that involves staggered treatment timing and we've seen some results where the P-value for pre-test of parallel trends assumption = 0. Does that mean that there's an error going on somewhere/we should not trust the regression output or is it just a very small p-value?

I'm happy to provide additional information if that would be helpful, thanks!

Best, Leigh

AnneSimon233 commented 3 months ago

Hello everyone, I'm glad other people are following up on this as well. I would still be happy to hear back about this because I could not fix the issue myself. Best, Anne