jkcshea / ivmte

An R package for implementing the method in Mogstad, Santos, and Torgovitsky (2018, Econometrica).
GNU General Public License v3.0
18 stars 2 forks source link

Specification test is not getting run for when `point = TRUE`? #139

Closed a-torgovitsky closed 4 years ago

a-torgovitsky commented 4 years ago

Doesn't appear to be

args <- list(data = ivmteSimData,
+              ivlike =  y ~ d + factor(z) + d*factor(z),
+              target = "ate",
+              m0 = ~ u,
+              m1 = ~ u,
+              m0.dec = TRUE,
+              m1.dec = TRUE,
+              propensity = d ~ factor(z),
+              bootstraps = 5,
+              point = TRUE,
+              noisy = TRUE)
> r <- do.call(ivmte, args)

LP solver: Gurobi ('gurobi')

Obtaining propensity scores...
Generating target moments...
    Integrating terms for control group...
    Integrating terms for treated group...
Generating IV-like moments...
    Moment 1...
    Moment 2...
    Moment 3...
    Moment 4...
    Moment 5...
    Moment 6...
    Moment 7...
    Moment 8...
    Moment 9...
    Moment 10...
    Moment 11...

Point estimate of the target parameter: -0.5949017

Bootstrap iteration 1...
    Point estimate:-0.5949011

Bootstrap iteration 2...
    Point estimate:-0.5949011

Bootstrap iteration 3...
    Point estimate:-0.5948301

Bootstrap iteration 4...
    Point estimate:-0.5949297

Bootstrap iteration 5...
    Point estimate:-0.5949011

--------------------------------------------------
Results 
--------------------------------------------------

Point estimate of the target parameter: -0.5949017

Number of bootstraps: 5

Bootstrapped confidence intervals (nonparametric):
    90%: [-0.5949297, -0.5948301]
    95%: [-0.5949297, -0.5948301]
    99%: [-0.5949297, -0.5948301]
p-value: 0

Warning message:
In (function (data, target, late.from, late.to, late.X, genlate.lb,  :
  If argument 'point' is set to TRUE, then shape restrictions on m0 and m1 are ignored, and the audit procedure is not implemented.
jkcshea commented 4 years ago

Ah, my mistake. I simply did not include it in the output. Here's what it is now:

--------------------------------------------------
Results 
--------------------------------------------------

Point estimate of the target parameter: -0.5949017

Number of bootstraps: 5

Bootstrapped confidence intervals (nonparametric):
    90%: [-0.5949297, -0.5948301]
    95%: [-0.5949297, -0.5948301]
    99%: [-0.5949297, -0.5948301]
p-value: 0

Bootstrapped J-test p-value: 0

Warning message:
If argument 'point' is set to TRUE, then shape 
restrictions on m0 and m1 are ignored, and the
audit procedure is not implemented. 

The user can also access results this way:

> r$jtest
Bootstrapped p-value          J-statistic              p-value 
          0.00000000          15.43732815           0.03078471 
                  df 
          7.00000000 

The last two elements correspond to the asymptotic J-test. Let me know if you'd like to get rid of that.

a-torgovitsky commented 4 years ago

What is the asymptotic J-test? Is it output from gmm?

jkcshea commented 4 years ago

Yep, that's what it is.

a-torgovitsky commented 4 years ago

Ok, so it's not accounting for the uncertainty in the propensity score I think it's fine to keep it in (probably helps for diagnostics), but maybe just mark what it is more clearly like "J-statistic (ignoring first step)" and "p-value (ignoring first step)"

jkcshea commented 4 years ago

Sure no problem. But just to make sure: isn't the J-statistic the same regardless of whether we include the first step?

a-torgovitsky commented 4 years ago

Ah yes sorry -- the test statistic is the same So just "p-value (ignoring first step)" is good

jkcshea commented 4 years ago

Great. Done!