HenrikBengtsson / doFuture

:rocket: R package: doFuture - Use Foreach to Parallelize via Future Framework
https://doFuture.futureverse.org
84 stars 6 forks source link

doFuture automatic export fails when the variable coinsides with a function argument name #46

Open CrossD opened 4 years ago

CrossD commented 4 years ago

doFuture fails to export the variable VAR when the it coinsides with a function argument name. The sessioninfo is appended.

> library(doFuture)
Loading required package: globals
Loading required package: future
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
> registerDoFuture()
>
> plan(multisession, workers  = 2)

>
> VAR <- 1
>
> foreach (i = 1:2) %dopar% {
+
+   f <- function(VAR=500) {
+     print(VAR)
+   }
+
+   f(VAR)
+ }
Error in { : task 1 failed - "object 'VAR' not found"
>
> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods
[8] base

other attached packages:
[1] doFuture_0.9.0   iterators_1.0.12 foreach_1.5.0    future_1.17.0
[5] globals_0.12.5

loaded via a namespace (and not attached):
[1] compiler_3.6.3   tools_3.6.3      listenv_0.8.0    codetools_0.2-16
[5] digest_0.6.25

However, the following works:

library(doFuture)
registerDoFuture()

plan(multisession, workers  = 2)

VAR <- 1

foreach (i = 1:2) %dopar% {

  print(VAR)
  f <- function(VAR=500) {
    print(VAR)
  }

  f(VAR)
}
HenrikBengtsson commented 4 years ago

Thank you for reporting. I can reproduce this, also with "plain" futures, e.g.

library(future)
plan(cluster, workers = 1L)
VAR <- 1
f <- future({ f <- function(VAR=500) { print(VAR) }; f(VAR) })
value(f)
## Error in print(VAR) : object 'VAR' not found
HenrikBengtsson commented 4 years ago

This is a problem with the globals package; I've created https://github.com/HenrikBengtsson/globals/issues/53 to track and solve it over there. I'll try to remember to report back here the day it is fixed.

HenrikBengtsson commented 3 years ago

Just an update: I thought I fixed this in globals 1.14.0 released yesterday but I forgot to actually validate it here. It turns out that I didn't fix it completely so it still doesn't work with foreach. Now tracking this in https://github.com/HenrikBengtsson/globals/issues/71.

CrossD commented 3 years ago

Good to know. I will follow the issue under globals. Thanks!