Check that the leveling-off test yields correct p-values

[Constantino-Carreto-Romero]

Per call: check that the leveling-off test yields correct p-values

[santiagohermo]

Thanks @Constantino-Carreto-Romero! Here is a small folder where I replicate what we saw in the call this week: issue132_stata.zip

(Happy to add it in a new branch if I can get write access @jorpppp @jmshapir)

To summarize the issue here, we used some data from the paper to run the following code:

clear all

import delimited "data_dynamic.csv"

xtset id t

foreach yvar in y_base y_smooth_m y_jump_m {

  xtevent `yvar' x_r, policyvar("z") panelvar("id") timevar("t") ///
    post(3) overidpost(1) pre(0) overidpre(3) norm(-1) ///
    reghdfe

  xteventplot, smpath(line) ///
    graphregion(color(white)) bgcolor(white)

  graph export STATA/`yvar'.png, replace width(1700) height(1214)
}

The data are available in the zip file.

This produces figures where the levelling-off p-value seems weird. In the call it was hypothesized that the p-value may actually correspond to a test of the last coefficient equal to zero.

Click to see figures

| outcome | figure | | --- | --- | | base |  | | smooth |  | | jump | 

---

I'll be able to look into this next week. I haven't explored xtevent's code that much, so I might be somewhat inefficient. fyi @jorpppp @jmshapir @SimonFreyaldenhoven

[jmshapir]

Thanks @santiagohermo! No need for you to investigate further just now. Let's see how @jorpppp prefers to proceed and we can take it from there.

[jorpppp]

Thanks @santiagohermo. I think there may be an issue with specifying leads and lags with overidpre and overidpost. I'll check it and fix here.

[jorpppp]

There was a bug in xteventtest when the leads and lags were specified using pre, overidpre, post, and overidpost, as opposed to using window. @SimonFreyaldenhoven was right: The test that was displayed was for the last coefficient being different from zero.

This bug did not affect the plots when the leads and lags were specified through the window option.

I fixed the bug in https://github.com/JMSLab/xtevent/commit/45cbc77c3fb742554a1cdb6830d349d40dc01dd7. Will start a PR.

[jorpppp]

Summary: In this issue we fixed a bug with the p-value for the leveling-off overidentification test when overidpostwas active.

Thread continues in #134