The right way to apply Placebo Tests

[LuMesserschmidt]

Dear colleagues,

Having completed my main analysis (see #89 ), I am currently working on checking for the parallel trend assumption. Doing so, I tried to make use of the placebotest command and ran into several issues.

Issue 1: Refinement with the outcome variable

To understand the effect of investments on nightlight development I am using previous nightlights as my refinement method. In your package instruction you write: „Additionally, you should not use the outcome variable in refinement when placebo.test = TRUE“. Does this mean I am not able to run a placebo test if I use the refinement method accordingly? Any way I can solve this issue?

Issue 2: Use t0 as a reference Ignoring issue 1 for a second, I ran the placebo test command and got a plot that showed treatment effects for t-2 and t-3, but not the effect for t-1. As I understand it, t-3 is calculated against t-1 and t-2 against t-1. But actually, I should calculate against t0, or how can I calculate the value of t-1 that would allow me to complete the plot? This is connected to issue 3 (see below).

Issue 3: Lag.in gives an error message I am not sure how the command lag.in in the placebotest function works / what it does as the description is not clear to me. What I understood is that it changes the reference time point from t-1 to X. I tried to play around a bit but the command always showed the following error: Object 'matchedsets' not found. Maybe you can provide a small example of how to use this function.

Thank you for your great package and for answering my questions!

[insongkim]

@LuMesserschmidt Thanks for the question.

1. The parallel trend is an identification assumption for the DiD. If you match on the outcome variable, then you are essentially removing the pre-treatment difference. We discuss this briefly in footnote 6 of our paper (please see here). In theory, you don't need a placebo test because you have already matched on the pre-treatment outcome. I hope this makes sense.
2. The placebo test will show the DiD in the outcomes between t-1 vs t-l, i.e., using t-1 as a reference. This is because t0 is the time of treatment.
3. I hope 2 makes sense. We will try to update the description when we update our package next time.

Thanks!

[LuMesserschmidt]

Dear In Song,

thanks a lot for your response, your answers definitely make sense, although I will need some time to reflect upon the first point. Is there any additional literature on the need for placebo tests for this scenario (matching on the outcome variable)? Otherwise, I will just - again - cite your paper 😄

Thanks again for this amazing package, it is so power- and helpful!

All the best,

Luca

[insongkim]

Thanks @LuMesserschmidt

Note that Difference-in-Differences (DiD) compares two differences between treated and control units, i.e., pre-treatment differences in outcomes vs. post-treatment differences in outcomes. If you match on pre-treatment outcomes then you don’t really need DiD if your quantity of interest is DiD. I hope that makes sense.

[LuMesserschmidt]

Dear @insongkim, that absolutely makes sense, thanks for clarifying this! This issue can be closed :)