JMSLab / xtevent

Stata package -xtevent-
MIT License
43 stars 12 forks source link

Check that the smoothest path's Wald critical value is correct #131

Closed Constantino-Carreto-Romero closed 1 year ago

Constantino-Carreto-Romero commented 1 year ago

per call: check that the smoothest path's Wald critical value is correct

jorpppp commented 1 year ago

It seems like we have the wrong degrees of freedom for the Wald critical value, as noted by @SimonFreyaldenhoven @santiagohermo

We need to subtract the number of normalized coefficients from the degrees of freedom.

jorpppp commented 1 year ago

@santiagohermo I am testing the fix for this issue. Can you please confirm where does the data that you were using for the tests comes from, please?:

Can you post a link to the test data you were using?

jmshapir commented 1 year ago

@santiagohermo would you be able to post a self-contained small ZIP archive with whatever data or code @jorpppp will need to reproduce the issue? That way we won't need to rely on links to an external repository here.

Thanks!

santiagohermo commented 1 year ago

Thanks @jorpppp @jmshapir! To reply to @jorpppp, the answer is yes. The data ultimately comes from the the data archive of the paper. The exact version I used is in .Rdata format and can be accessed in df_sample_dynamic.zip. I transformed it into csv here: df_sample_dynamic_csv.zip.

To conduct the analysis we discussed in the call I made some changes to my local xteventplot.ado:

  1. I dropped the cap qui in this line
  2. I added the following lines right after the line above:

    mata: st_matrix("param",param)
    di _n "Smothest path:"
    mat list param
    
    mata: st_matrix("WB",WB)
    di _n "Optimal Wald value"
    mat list WB
  3. Then I ran stata from the cmd to get a log file with WB values.

Let me know if I can be of any more help @jorpppp!

jorpppp commented 1 year ago

In https://github.com/JMSLab/xtevent/commit/e7056c4d6087a3fb496c3ec86e3535e01e33feb9 and https://github.com/JMSLab/xtevent/commit/fff916b616293a37d0c81456cf17f4219aa01792 I fixed the critical value and I can confirm that now the critical value is the one corresponding to the number of estimated parameters (omitting the normalized parameters).

I also added mata saves for the Wald critical value and the Wald optimal value in case we need further debugging later. I'll start a PR now.

jorpppp commented 1 year ago

Summary: In this issue we fixed the critical value that defines the Wald region where the smoothest path has to lie.

Thread continues in #135