Confidence interval of aggregate effects off when using fe = 'none'

fhollenbach commented 2 years ago

Sorry to be a pain and submit another issue, but I wanted to keep track of it. I just realized when replying to the other issue that the confidence interval of the aggregate effects is weirdly off when using fe = 'none'.

Based on the example mpdta:

mod = 
  etwfe(
   fml  = lemp ~ lpop, # outcome ~ controls
   tvar = year,        # time variable
   gvar = first.treat, # group variable
   data = mpdta,       # dataset
   vcov = ~countyreal, # vcov adjustment (here: clustered)
   fe = "none"
   )

emfx(mod, type = "simple")
     Term    Contrast .Dtreat   Effect Std. Error z value   Pr(>|z|) 2.5 % 97.5 %
1 .Dtreat mean(dY/dX)       1 -0.05063     0.0125  -4.051 5.0988e-05 8.418  8.694

emfx(mod, type = "event")
     Term    Contrast event   Effect Std. Error z value   Pr(>|z|) 2.5 % 97.5 %
1 .Dtreat mean(dY/dX)     0 -0.03321    0.01337  -2.485 0.01296119 8.418  8.694
2 .Dtreat mean(dY/dX)     1 -0.05735    0.01715  -3.344 0.00082628 5.871  6.147
3 .Dtreat mean(dY/dX)     2 -0.13787    0.03079  -4.478 7.5344e-06 5.163  5.462
4 .Dtreat mean(dY/dX)     3 -0.10954    0.03232  -3.390 0.00069966 5.264  5.513

     Term    Contrast first.treat   Effect Std. Error z value   Pr(>|z|) 2.5 % 97.5 %
1 .Dtreat mean(dY/dX)        2004 -0.08763    0.02305  -3.802 0.00014351 5.283  5.557
2 .Dtreat mean(dY/dX)        2006 -0.02128    0.01859  -1.145 0.25240119 5.895  6.162
3 .Dtreat mean(dY/dX)        2007 -0.04595    0.01797  -2.557 0.01055532 8.418  8.694

I'll try to also take a look at this over the weekend.

grantmcdermott commented 2 years ago

Oh, definitely something wrong there. I’ll mark to take a look when I get time too.

vincentarelbundock commented 2 years ago

Yep, this was my bug.

Should be fixed in the Github version of marginaleffects

grantmcdermott commented 2 years ago

Fix confirmed. Merci, Vincent.

I'll wait until the dev version hits CRAN and then submit etwfe too.

data("mpdta", package = "did")  

library(etwfe)

mod = 
  etwfe(
   fml  = lemp ~ lpop, # outcome ~ controls
   tvar = year,        # time variable
   gvar = first.treat, # group variable
   data = mpdta,       # dataset
   vcov = ~countyreal, # vcov adjustment (here: clustered)
   fe = "none"
   )

emfx(mod, type = "simple")
#>      Term    Contrast .Dtreat   Effect Std. Error z value   Pr(>|z|)    2.5 %
#> 1 .Dtreat mean(dY/dX)       1 -0.05063     0.0125  -4.051 5.0988e-05 -0.07512
#>     97.5 %
#> 1 -0.02613
#> 
#> Model type:  etwfe 
#> Prediction type:  response

emfx(mod, type = "event")
#>      Term    Contrast event   Effect Std. Error z value   Pr(>|z|)    2.5 %
#> 1 .Dtreat mean(dY/dX)     0 -0.03321    0.01337  -2.485 0.01296119 -0.05941
#> 2 .Dtreat mean(dY/dX)     1 -0.05735    0.01715  -3.344 0.00082628 -0.09096
#> 3 .Dtreat mean(dY/dX)     2 -0.13787    0.03079  -4.478 7.5344e-06 -0.19821
#> 4 .Dtreat mean(dY/dX)     3 -0.10954    0.03232  -3.390 0.00069966 -0.17288
#>      97.5 %
#> 1 -0.007015
#> 2 -0.023733
#> 3 -0.077526
#> 4 -0.046203
#> 
#> Model type:  etwfe 
#> Prediction type:  response

^{Created on 2022-11-06 with reprex v2.0.2}

grantmcdermott / etwfe

Confidence interval of aggregate effects off when using fe = 'none' #8