Comparing did2s with the gmm approach

[friosavila]

Hi Kyle, I was looking into the stata-command for did2s, and I found 1 problem trying to replicilate your results.

1. the did2s that is available from ssc seems incomplete. I may suggest to update it with the version from github.
2. I was trying to replicate the example described in the blog https://causalinf.substack.com/p/two-stage-did-and-taming-the-did

The results replicate fine, however, they do not match with the gmm version

gmm (eq1: (l_homicide-{xb: i.year}-{xg: ibn.sid})(1-post)) /// (eq2: l_homicide-{xb:} - {xg:} - {delta}post) [pw=popwt], /// instruments(eq1: i.year ibn.sid) /// instruments(eq2: post) winitial(identity) /// onestep quickderivatives vce(cluster sid)

Point estimates are the same, but standard errors are off. What may explain this?

[kylebutts]

The standard errors I use are analytical and from 2-step estimator asymptotic theory. The gmm command uses analytic derivatives (quickderivatives does this) which are approximations. These two reasons are why the se's differ

[friosavila]

In that case, however, which one is correct? Playing around with block bootstrap and those standard errors are similar to what gmm produces. Saying it in a different way, is there a way to get gmm produce what did2s is generating?

[kylebutts]

Let me look into it a bit more, but I suspect the gmm is not doing something right as the standard errors are pretty close to the manual approach which doesn't account for the first-stage at all.

[friosavila]

I dont think its gmm. I tried using an ML approach. The point estimates are a bit different, but the standard errors and t-stats are closer to the gmm method. Here is the code that I used if you are interested.

capture program drop ml2
program ml2
    args lnf delta xt xi
    qui {
        replace `lnf'=-(1-$ML_y2)*($ML_y1-`xt'-`xi')^2
        replace `lnf'=`lnf'-($ML_y1-`xt'-`xi'-`delta'*$ML_y2)^2
    }
end
use https://github.com/scunning1975/mixtape/raw/master/castle.dta, clear

ml model lf ml2 (l_homicide post=) (xt: = i.year, nocons) (xi: = ibn.sid, nocons) [pw=popwt], cluster(sid) maximize
ml display

**with the outcome of interest:

------------------------------------------------------------------------------
             |               Robust
             | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
eq1          |
       _cons |   .0753188   .0340749     2.21   0.027     .0085333    .1421043
-------------+----------------------------------------------------------------

[friosavila]

Up to closer inspection, I think the problem has to do with how weights are being treated in the segment of your Mata code. When you drop weights, the results are almost identical.

[kylebutts]

Can you show me where in the code you dropped weights? Thanks for the help by the way. This code is still very much in beta

[friosavila]

Of course! and Yes, I m also "debugging" -csdid- often. Here the code:

gmm (eq1: (l_homicide-{xb: i.year}-{xg: ibn.sid})*(1-post)) ///
(eq2: l_homicide-{xb:} - {xg:} - {delta}*post) , ///
instruments(eq1: i.year ibn.sid) ///
instruments(eq2: post) winitial(identity) ///
onestep quickderivatives vce(cluster sid)

Since this doesn't use the sample weights produces the same as (off by a hair) your command.

gmm

-------------+----------------------------------------------------------------
      /delta |   .0668998   .0570145     1.17   0.241    -.0448465    .1786462
------------------------------------------------------------------------------

did2s l_homicide, first_stage(i.sid i.year) second_stage(i.post) treatment(post) cluster(sid)

             |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.post |   .0668998   .0566979     1.18   0.238     -.044226    .1780257

and did_imputation
** after changing treatment_date=treatment_date+1
did_imputation l_homicide sid year treatment_date

------------------------------------------------------------------------------
  l_homicide |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         tau |   .0655189   .0550215     1.19   0.234    -.0423212     .173359
------------------------------------------------------------------------------

which is also quite close the ML code I show you before

-------------+----------------------------------------------------------------
eq1          |
       _cons |   .0682523    .056343     1.21   0.226     -.042178    .1786825
-------------+----------------------------------------------------------------

[kylebutts]

I was definitely doing weighting wrong with the standard error adjustment. Thanks for pointing out, going to try and fix this weekend!

[kylebutts]

Thanks for pointing this out. I have it fixed now and it produce the correct results!

  use "https://github.com/scunning1975/mixtape/raw/master/castle.dta", clear
  did2s l_homicide [pw=popwt], first_stage(i.year i.sid) second_stage(post) treatment(post) cluster(sid)
                                      (Std. Err. adjusted for clustering on sid)
  ------------------------------------------------------------------------------
               |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
  -------------+----------------------------------------------------------------
          post |   .0751416   .0354273     2.12   0.034     .0057053     .144578
  ------------------------------------------------------------------------------