Handling of universal base period

[simonschoe]

Hi there,

maybe the question is trivial but I couldn't really figure it out by reading the paper and README of the repo: How would I specify numPrePeriods and numPostPeriods in the call to createSensitivityResults_relativeMagnitudes when my base period is some pre-treatment period other the canoncial "-1"?

Let's say my event study estimates range from -8 to 8, with 0 being the first treatment period and having defined -4 as the reference period to which the estimates are normalized (due to anticipation effects). Would I then define numPrePeriods = 4 (-8,-7,-6,-5) or numPrePeriods = 7 (-8,-7,-6,-5,-3,-2,-1). I guess my issue here is that in the second case pre-treatment trends would be drawn from -5 to -3 right?

Grateful for any hints!

[jonathandroth]

Hi Simon,

Let me comment on two things: first, mechanically how you would do this; and second, whether you should want to do this.

On the first point, the defaults in HonestDID assume that the last pre-treatment period is normalized to zero. There is not currently an option to just change the default base period. There probably should be, and I will add this to our list of things to do when we have time for package improvements. One can pass their own matrix A to HonestDiD; so if you wanted to do this yourself, you could modify the code for e.g. create_A_rm here https://github.com/asheshrambachan/HonestDiD/blob/master/R/deltarm.R to impose that the fourth pre-period is zero instead of the last one, although admittedly that's kind of annoying to do manually. Sorry.

On the second point, HonestDID imposes bounds on the post-treatment violations of parallel trends as a function of the pre-treatment violations. For example, the "relative magnitudes" (RM) restriction with M=1 says that the magnitude of the post-treatment violation is no larger than the largest pre-treatment violation. But if you think that there's anticipation in some pre-treatment periods, it is sort of weird to use those periods for this calculation. Then you're saying that the post-treatment bias is no larger than the sum of the pre-treatment violations and pre-treatment anticipatory effects. Put otherwise, if you think there's anticipation in the 3 periods before treatment, then you're effectively saying that the start-date of the treatment effects is at -3. So conceptually it might make more sense to use HonestDID only using the "clean" pre-periods without anticipation.

Best, Jon

On Thu, Jul 20, 2023 at 7:59 AM Simon Schölzel @.***> wrote:

Hi there,

maybe the question is trivial but I couldn't really figure it out by reading the paper and README of the repo: How would I specify numPrePeriods and numPostPeriods in the call to createSensitivityResults_relativeMagnitudes when my base period is some pre-treatment period other the canoncial "-1"?

Let's say my event study estimates range from -8 to 8, with 0 being the first treatment period and having defined -4 as the reference period to which the estimates are normalized (due to anticipation effects). Would I then define numPrePeriods = 4 (-8,-7,-6,-5) or numPrePeriods = 7 (-8,-7,-6,-5,-3,-2,-1). I guess my issue here is that in the second case pre-treatment trends would be drawn from -5 to -3 right?

Grateful for any hints!

— Reply to this email directly, view it on GitHub https://github.com/asheshrambachan/HonestDiD/issues/34, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE6EXFAIEPVZUQBPLIZLAJ3XREMQNANCNFSM6AAAAAA2RJ6WUQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jonathandroth]

@mcaceresb -- when you have RA time in the fall, one thing to put on your to-do list is to add a referencePeriod option (with default =-1), and to update the create_A_XX functions to impose base period of referencePeriod instead of -1. Thanks!

[simonschoe]

Hi Jonathan, thanks for taking the time to reply, I appreciate it.

On the second point, HonestDID imposes bounds on the post-treatment violations of parallel trends as a function of the pre-treatment violations. For example, the "relative magnitudes" (RM) restriction with M=1 says that the magnitude of the post-treatment violation is no larger than the largest pre-treatment violation. But if you think that there's anticipation in some pre-treatment periods, it is sort of weird to use those periods for this calculation. Then you're saying that the post-treatment bias is no larger than the sum of the pre-treatment violations and pre-treatment anticipatory effects. Put otherwise, if you think there's anticipation in the 3 periods before treatment, then you're effectively saying that the start-date of the treatment effects is at -3. So conceptually it might make more sense to use HonestDID only using the "clean" pre-periods without anticipation.

So yes, thats exactly what I would like to do. I'd like to assess the sensitivity of the treatment effect estimate wrt violations in pre-treatment trends from -8 till -4, i.e., the pre-treatment period where we expect no anticipation.

Is modifying .create_A_RM the only feasible option here or could i also achieve the same thing by subsetting the betahat vector and sigma matrix that serve as inputs to createSensitivityResults_relativeMagnitudes?

[jonathandroth]

If your goal is only to use those periods then I think you can use HonestDID as usual, just pretending everything after -4 is a treated period. The suggestion I had was if you wanted to count the later pre-periods as pre-treatment despite having normalized -4 to 0

On Sat, Jul 22, 2023, 8:18 AM Simon Schölzel @.***> wrote:

Hi Jonathan, thanks for taking the time to reply, I appreciate it.

On the second point, HonestDID imposes bounds on the post-treatment violations of parallel trends as a function of the pre-treatment violations. For example, the "relative magnitudes" (RM) restriction with M=1 says that the magnitude of the post-treatment violation is no larger than the largest pre-treatment violation. But if you think that there's anticipation in some pre-treatment periods, it is sort of weird to use those periods for this calculation. Then you're saying that the post-treatment bias is no larger than the sum of the pre-treatment violations and pre-treatment anticipatory effects. Put otherwise, if you think there's anticipation in the 3 periods before treatment, then you're effectively saying that the start-date of the treatment effects is at -3. So conceptually it might make more sense to use HonestDID only using the "clean" pre-periods without anticipation.

So yes, thats exactly what I would like to do. I'd like to assess the sensitivity of the treatment effect estimate wrt violations in pre-treatment trends from -8 till -4, i.e., the pre-treatment period where we expect no anticipation.

Is modifying .create_A_RM the only feasible option here or could i also achieve the same thing by subsetting the betahat vector and sigma matrix that serve as inputs to createSensitivityResults_relativeMagnitudes?

— Reply to this email directly, view it on GitHub https://github.com/asheshrambachan/HonestDiD/issues/34#issuecomment-1646570615, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE6EXFDSD3PZ6LT67F47II3XRPAJBANCNFSM6AAAAAA2RJ6WUQ . You are receiving this because you commented.Message ID: @.***>