futureverse / future.apply

:rocket: R package: future.apply - Apply Function to Elements in Parallel using Futures
https://future.apply.futureverse.org
211 stars 16 forks source link

future_mapply tries to subset MoreArgs if future.seed = TRUE #51

Closed mllg closed 4 years ago

mllg commented 4 years ago

Example:

library(future.apply)

future_mapply(
  function(x, y) x,
  x = 1:3, MoreArgs = list(y = identity),
  future.seed = TRUE
)
HenrikBengtsson commented 4 years ago

Thanks, I can confirm;

y0 <- base::mapply(
  function(x, y) y,
  x = 1:2, MoreArgs = list(y = 3:4)
)
print(y0)

y1 <- future.apply::future_mapply(
  function(x, y) y,
  x = 1:2, MoreArgs = list(y = 3:4),
  future.seed = FALSE
)
stopifnot(identical(y1, y0))

y2 <- future.apply::future_mapply(
  function(x, y) y,
  x = 1:2, MoreArgs = list(y = 3:4),
  future.seed = TRUE
)
print(y2)
stopifnot(identical(y2, y0))

gives:

> print(y0)
     [,1] [,2]
[1,]    3    3
[2,]    4    4

> print(y2)
[1] 3 4

> stopifnot(identical(y2, y0))
Error: identical(y2, y0) is not TRUE
HenrikBengtsson commented 4 years ago

There's more to it. The above is with plan(sequential), with parallelization there's (luckily) a sanity-check error kicking in;

> plan(multisession, workers = 2L)
> y2 <- future.apply::future_mapply(
+   function(x, y) y,
+   x = 1:2, MoreArgs = list(y = 3:4),
+   future.seed = TRUE
+ )
Error: Unexpected error in future_mapply(): After gathering and merging the
values from 2 chunks in to a list, the total number of elements (= 4) does not
match the number of input elements in 'X' (= 2). There were in total 2 chunks
and 2 elements (2 chunks with 2 elements). Example of the first few values: 
List of 3\n $ : int 3\n $ : int 4\n $ : int 3
HenrikBengtsson commented 4 years ago

Fixed in the develop branch.