Odd error with future.apply after loading BiocGenerics

[DarwinAwardWinner]

Certain functions seem to no longer work with future_lapply after being transformed into S4 generics by BiocGenerics, e.g. table. The problem seems limited to future.apply functions, because table still works inside future.

library(future)
library(future.apply)
x <- replicate(10, sample(letters, size = 100, replace = TRUE), simplify = FALSE)
y <- future_lapply(x, table) # This works before loading BiocGenerics
library(BiocGenerics)
#> Loading required package: parallel
#> 
#> Attaching package: 'BiocGenerics'
#> The following objects are masked from 'package:parallel':
#> 
#>     clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
#>     clusterExport, clusterMap, parApply, parCapply, parLapply,
#>     parLapplyLB, parRapply, parSapply, parSapplyLB
#> The following objects are masked from 'package:stats':
#> 
#>     IQR, mad, sd, var, xtabs
#> The following objects are masked from 'package:base':
#> 
#>     anyDuplicated, append, as.data.frame, basename, cbind, colnames,
#>     dirname, do.call, duplicated, eval, evalq, Filter, Find, get, grep,
#>     grepl, intersect, is.unsorted, lapply, Map, mapply, match, mget,
#>     order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,
#>     rbind, Reduce, rownames, sapply, setdiff, sort, table, tapply,
#>     union, unique, unsplit, which.max, which.min
y <- future_lapply(x, table) # this fails after loading BiocGenerics
#> Error in getGlobalsAndPackages(expr, envir = envir, globals = globals): Did you mean to create the future within a function?  Invalid future expression tries to use global '...' variables that do not exist: FUN()
y <- future_lapply(x, base::table) # This works
y <- value(future(table(x[[1]]))) # This works

^{Created on 2021-04-22 by the reprex package (v2.0.0)}

This is obviously a problem since pretty much every Bioconductor package loads BiocGenerics.

[HenrikBengtsson]

Thanks for reporting. I can reproduce this with both future.apply::future_lapply() and furrr::future_map(), which suggests a bug in future::getGlobalsAndPackages() or possible higher up in the globals package.

[HenrikBengtsson]

Some good news: This is related to https://github.com/HenrikBengtsson/globals/issues/72 and I've verified that @DavisVaughan's suggestion there solves the problem reported here. Thus, this will be solved soon. What needs to be figured out is whether it should be fixed in globals or future, which is a design decision. After that, the fix needs to validated against all reverse package dependencies. Then I'll roll out the fix.

So, stay tuned. I'll keep this issue open to remind me to add a package test for this bug.

[HenrikBengtsson]

This has now been fixed in the next relase of the future package (to become future 1.22.0). The following now works as expected:

library(future.apply)
X <- list(sample(letters, size = 100, replace = TRUE))
y1 <- future_lapply(X, FUN = table)
library(BiocGenerics)
y2 <- future_lapply(X, FUN = table)
stopifnot(identical(y2, y1))

Thanks for reporting.