jkcshea / ivmte

An R package for implementing the method in Mogstad, Santos, and Torgovitsky (2018, Econometrica).
GNU General Public License v3.0
18 stars 2 forks source link

Remove everything related to seeds; we don't do that anymore! #188

Closed a-torgovitsky closed 4 years ago

a-torgovitsky commented 4 years ago

ivmte no longer screws up the random number generator

devtools::load_all()

data <- read.csv("./mwe1.csv")

args <- list(data = data, ivlike = y ~ d*factor(z),
             target = "ate",
             m0 = ~ 1,
             m1 = ~ u,
             m0.lb = 0,
             m0.ub = 0,
             propensity = d ~ factor(z))

for (i in 1:3) {
    print(rnorm(1))
    do.call(ivmte, args)
}

gives

[1] 0.2428141
[1] -0.1794061
[1] -0.6305186

And we still get reproducibility by setting the seed before calling it

# audit is stable
set.seed(2)
print(do.call(ivmte, args)$bounds)

set.seed(2)
print(do.call(ivmte, args)$bounds)

# bootstrap also stable
args[["bootstraps"]] <- 20
set.seed(2)
r <- do.call(ivmte, args)
print(r$bounds.ci)

set.seed(2)
r <- do.call(ivmte, args)
print(r$bounds.ci)

yields

[1] 0.2727059 0.2727059
[1] 0.2727059 0.2727059
$backward
     lb.backward ub.backward
0.9    0.1117307   0.5199328
0.95   0.1117307   0.5202089
0.99   0.1117307   0.5202089

$forward
     lb.forward ub.forward
0.9  0.02547898  0.4336811
0.95 0.02520293  0.4336811
0.99 0.02520293  0.4336811

$backward
     lb.backward ub.backward
0.9    0.1117307   0.5199328
0.95   0.1117307   0.5202089
0.99   0.1117307   0.5202089

$forward
     lb.forward ub.forward
0.9  0.02547898  0.4336811
0.95 0.02520293  0.4336811
0.99 0.02520293  0.4336811
jkcshea commented 4 years ago

Looks good! It looks like you caught all cases where seeds were being declared, and you deleted all those seed-related objects (e.g. bseeds for the boostrap).

The testthat files continue to run just fine since the seed was manually set before calling ivmte.

Thanks for taking care of this!

a-torgovitsky commented 4 years ago

Great, thanks!