DavisVaughan / furrr

Apply Mapping Functions in Parallel using Futures
https://furrr.futureverse.org/
Other
695 stars 39 forks source link

Use of variables in glue() inside furrr loops #262

Closed MS-equinom closed 1 year ago

MS-equinom commented 1 year ago

Hi

When the first time a global variable appears in a furrr function is in a glue expression, the loop fails with error object XXX not found.

Example:

library(furrr)
plan(multisession, workers = 2)
options(future.scheduling = 2L)

t1<- "sdsdsd"
furrr::future_walk(1:4,.f = function(x) {
    t2 <- glue::glue("{t1}")
    print(t2)
    })

However, adding print(t1) anywhere in the loop will make the code work.

I assume this is because furrr shares with the workers only those global variables that are used in the loop, and that it does not recognize variables inside glue expressions as variables to be shared.

Thanks for the awesome furrr package. Love it.

DavisVaughan commented 1 year ago

it does not recognize variables inside glue expressions as variables to be shared.

Yea, this is exactly right. Since it is embedded in a string the {globals} package can't "find" that t1 needs to be exported.

This is actually listed as one of the "common gotchas" https://future.futureverse.org/articles/future-4-issues.html#glueglue---object-not-found

If you add t1 manually through furrr_options() then it will find it

furrr::future_walk(1:4, .f = function(x) {
  t2 <- glue::glue("{t1}")
  print(t2)
}, .options = furrr::furrr_options(globals = "t1"))