HenrikBengtsson / doFuture

:rocket: R package: doFuture - Use Foreach to Parallelize via Future Framework
https://doFuture.futureverse.org
84 stars 6 forks source link

plyr: RNG warnings #61

Closed HenrikBengtsson closed 1 year ago

HenrikBengtsson commented 3 years ago

(was Issue #58)

doFuture::registerDoFuture()
y <- plyr::llply(1:2, rnorm, .parallel = TRUE)

Warning messages:
1: In setup_parallel() : No parallel backend registered
2: UNRELIABLE VALUE: One of the foreach() iterations ('doFuture-1') unexpectedly generated random numbers without declaring so. There is a risk that those random numbers are not statistically sound and the overall results might be invalid. To fix this, use '%dorng%' from the 'doRNG' package instead of '%dopar%'. This ensures that proper, parallel-safe random numbers are produced via the L'Ecuyer-CMRG method. To disable this check, set option 'future.rng.onMisuse' to "ignore".

This is a true RNG mistake. This happens because plyr does not use doRNG. Also, looking at the source code, there are no other attempts to use parallel-safe RNG.

Workaround: After doFuture::registerDoFuture(), call doRNG::registerDoRNG(), which will automatically turn all %dopar% to %dorng%, e.g.

doFuture::registerDoFuture()
doRNG::registerDoRNG()
y <- plyr::llply(1:2, rnorm, .parallel = TRUE)

To disable the RNG warnings, set:

options(future.rng.onMisuse = "ignore")

## Slightly better: in doFuture (>= 0.12.0) [next release]
options(doFuture.rng.onMisuse = "ignore")
HenrikBengtsson commented 1 year ago

Added withDoRNG() to the next release, e.g.

library(doFuture)
registerDoFuture()
plan(multisession)
y <- withDoRNG(plyr::llply(1:2, rnorm, .parallel = TRUE))