search

futureverse / future.apply

:rocket: R package: future.apply - Apply Function to Elements in Parallel using Futures
https://future.apply.futureverse.org
211 stars 16 forks source link

options("test"=5) are not global #16

Closed Rloveeee closed 6 years ago

Rloveeee commented 6 years ago

I think I found a small bug in the future.apply package:

options("test"=5) test=4 plan(multiprocess, workers = 12) FUN=function(x) c(getOption("test"),test) future.apply::future_lapply(list(1,2),FUN=FUN,future.globals=TRUE) future.apply::future_lapply(list(1,2),FUN=FUN,future.globals=FALSE) lapply(list(1,2),FUN=FUN)

It seems that future.apply::future_lapply can't recover getOptions( ) stuff which are a sort of global variable.

Best

HenrikBengtsson commented 6 years ago

Hi. Yes, automatic handling of options would be useful.

Unfortunately, we can't just pass all options on to futures as is. For example, some options are specific to the current R session/compute system (remember that futures can be resolved anywhere - also on a computer across the world). Other options are set conditionally on environment variables, which again, may be set differently depending on the computer where the future runs - we need to make sure that is also respected.

We've got https://github.com/HenrikBengtsson/future/issues/134 that is dedicated to this topic. Note that there is also an example how you can "pass" options yourself already now.

I'm closing this one - please continue the discussion there, e.g. if you have further ideas or thoughts. When we've figured out a way to support options in the core Future API, which probably will start off as a manual options argument, then we'll probably add a corresponding future.options argument to the functions of the future.apply package.