amices / mice

Multivariate Imputation by Chained Equations
https://amices.org/mice/
GNU General Public License v2.0
428 stars 107 forks source link

Revert internal seed behaviour back to mice 3.13.10. #515

Closed gerkovink closed 1 year ago

gerkovink commented 1 year ago

Restores seed behaviour to mice <3.14.4. Also closes #514.

# Must be equal
library(mice, warn.conflicts = FALSE)
A <- futuremice(nhanes, n.core = 5, parallelseed = 123)
B <- futuremice(nhanes, n.core = 5, parallelseed = 123)
identical(A$imp, B$imp)
#> [1] TRUE
identical(A[-11], B[-11])
#> [1] TRUE
all(complete(A, "long") == complete(B, "long"))
#> [1] TRUE

#Must be different
C <- futuremice(nhanes, n.core = 5)
D <- futuremice(nhanes, n.core = 5)
identical(C$imp, D$imp)
#> [1] FALSE
identical(C[-11], D[-11])
#> [1] FALSE
all(complete(C, "long") == complete(D, "long"))
#> [1] FALSE

# Should be equal
set.seed(123)
E <- futuremice(nhanes, n.core = 5)
set.seed(123)
F <- futuremice(nhanes, n.core = 5)
identical(E$imp, F$imp)
#> [1] TRUE
identical(E[-11], F[-11])
#> [1] TRUE
all(complete(E, "long") == complete(F, "long"))
#> [1] TRUE

# testthat: mice
library(testthat)
context("mice: seed")

imp1 <- mice(nhanes2, m = 2, seed = 123, print = FALSE)
imp2 <- mice(nhanes2, m = 2, seed = 123, print = FALSE)
test_that("same seed yields same imputations", {
  expect_identical(imp1[-11], imp2[-11])
})
#> Test passed 😸

# testthat: futuremice
context("futuremice: parallelseed")

imp1 <- futuremice(nhanes2, m = 4, n.core = 2, parallelseed = 1)
imp2 <- futuremice(nhanes2, m = 4, n.core = 2, parallelseed = 1)
test_that("same parallelseed yields same imputations", {
  expect_identical(imp1[-11], imp2[-11])
})
#> Test passed 🎊

Created on 2022-11-09 with reprex v2.0.2

stefvanbuuren commented 1 year ago

Thanks. This is useful, but we need to go a little further back, so that we have the same behaviour as mice 3.14.0, currently latest on CRAN. This would mean de-implement all withr functions, and restore the older global set.seed() approach from mice 3.14.0. When the prospective 3.15.0 comes out on CRAN, the hope is that CRAN users then won't experience a difference in the imputed values.

gerkovink commented 1 year ago

I believe we need to go a bit further back, as mice 3.14.0 already uses withr. I believe it is 3.13.6. will do so later today

stefvanbuuren commented 1 year ago

I introduced the unfortunate local randomised in PR #432.

stefvanbuuren commented 1 year ago

For cross-referencing

Added local random seed: #426, #432 Problems with local random seed: #459, #492, #502, #505

gerkovink commented 1 year ago

@stefvanbuuren I did revert #432. Only mentions of withr in readme.md and - of course - news.md.