Open HenrikBengtsson opened 1 year ago
My guess is that this is caused by the use of Rcpp::RNGScope
in the auto-generated file RcppExports.cpp:
https://github.com/r-lib/later/blob/ba70887d77527e5647e375013e2e1ad9dc2c3646/src/RcppExports.cpp
There's one more use of RNGScope
in later.cpp:
https://github.com/r-lib/later/blob/ba70887d77527e5647e375013e2e1ad9dc2c3646/src/later.cpp#L194
I believe those are the only places where later interacts with R's random number generator.
Here's a simple example with just Rcpp:
str(globalenv()$.Random.seed)
#> NULL
Rcpp::cppFunction('int go() { return 1; }')
str(globalenv()$.Random.seed)
#> NULL
go()
#> [1] 1
str(globalenv()$.Random.seed)
#> int [1:626] 10403 624 -1688476694 -149789597 758540872 1648411561 -1149942954 1420315103 -1604158828 -374756219 ...
It looks like when the Rcpp-wrapped C++ function is run, that causes the random seed to be set. The later package runs some of its C++ functions on load, and that's probably what's causing the seed to be set.
Note that if the random seed is already set (to a non-NULL value) before loading later, then it does not alter the seed:
rnorm(1)
#> [1] -0.484127
str(globalenv()$.Random.seed)
#> int [1:626] 10403 2 -1539884961 -1138419659 1420516300 1450512162 -1641721475 806492791 -2033368162 541053528 ...
Rcpp::cppFunction('int go() { return 1; }')
str(globalenv()$.Random.seed)
#> int [1:626] 10403 2 -1539884961 -1138419659 1420516300 1450512162 -1641721475 806492791 -2033368162 541053528 ...
go()
#> [1] 1
str(globalenv()$.Random.seed)
#> int [1:626] 10403 2 -1539884961 -1138419659 1420516300 1450512162 -1641721475 806492791 -2033368162 541053528 ...
Issue
Loading later causes the RNG state to be updated, e.g.
Why is this a problem? This makes it near impossible to get numerically reproducible results in parallel processing where the persistent workers are used (e.g.
parallel::makeCluster()
), because the results will depend on a package dependencies already being loaded or not. This affects all packages importing later directly or indirectly.The only workaround for this is to (i) know what packages might be loaded up-front, and (ii) pre-load them all on the parallel workers before performing the actually tasks. In practice, that's not feasible.
Suggestion
I don't know what the RNG is used for during
.onLoad()
, but could one solution be do draw the random number in stealth mode, i.e. make sure to undo.Random.seed
afterward?Details
It's not one of the package dependencies that forwards the RNG;