R_FUTURE_SUPPORTSMULTICORE_UNSTABLE = "quiet": Package 'future' is not installed on worker error

[jfiksel]

When I use plan(multisession) I get the error message:

Error: Initialization of plan() failed, because the test future used for validation failed. The reason was: Package 'future' is not installed on worker (r_version: 4.1.0 (2021-05-18); platform: x86_64-apple-darwin17.0 (64-bit); os: Darwin 19.6.0 Darwin Kernel Version 19.6.0: Tue Nov 10 00:10:30 PST 2020; root:xnu-6153.141.10~1/RELEASE_X86_64; hostname: MacBook-Pro.local)

I'm using a MacBook Pro with macOS Catalina. Below is the session info.

R version 4.1.0 (2021-05-18) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS Catalina 10.15.7

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] future.apply_1.7.0 future_1.21.0

loaded via a namespace (and not attached): [1] compiler_4.1.0 parallelly_1.27.0 cli_3.0.1 parallel_4.1.0 tools_4.1.0
[6] listenv_0.8.0 codetools_0.2-18 digest_0.6.27 globals_0.14.0 rlang_0.4.11

[jfiksel]

Just a follow-up: I only get this error message after running the line Sys.setenv(R_FUTURE_SUPPORTSMULTICORE_UNSTABLE = "quiet")

[HenrikBengtsson]

Thank you. I can reproduce this in vanilla R on Linux with plan(multisession) and plan(cluster);

$ R --quiet --vanilla
> library(future)
> Sys.setenv(R_FUTURE_SUPPORTSMULTICORE_UNSTABLE = "quiet")
> plan(cluster, workers = 1L)
Error: Initialization of plan() failed, because the test future used for validation failed. The reason was: Package 'future' is not installed on worker (r_version: 4.1.0 Patched (2021-07-20 r80647); platform: x86_64-pc-linux-gnu (64-bit); os: Linux 5.4.0-77-generic #86~18.04.1-Ubuntu SMP Fri Jun 18 01:23:22 UTC 2021; hostname: hb-x1)

> sessionInfo()
R version 4.1.0 Patched (2021-07-20 r80647)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.5 LTS

Matrix products: default
BLAS:   /home/hb/software/R-devel/R-4-1-branch/lib/R/lib/libRblas.so
LAPACK: /home/hb/software/R-devel/R-4-1-branch/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] future_1.21.0

loaded via a namespace (and not attached):
[1] compiler_4.1.0    parallelly_1.27.0 tools_4.1.0       parallel_4.1.0   
[5] listenv_0.8.0     codetools_0.2-18  digest_0.6.27     globals_0.14.0

[HenrikBengtsson]

It turns out this is a bug in parallelly. The startup code called when parallelly is loaded attempts to coerce R_FUTURE_SUPPORTSMULTICORE_UNSTABLE to a logical value, whereas it should be a character string. This results in an error when parallelly is loaded;

> Sys.setenv(R_FUTURE_SUPPORTSMULTICORE_UNSTABLE = "quiet")
> library(future)
Error: .onLoad failed in loadNamespace() for 'parallelly', details:
  call: update_package_option("supportsMulticore.unstable", mode = "logical", 
  error: Coercing environment variable 'R_PARALLELLY_SUPPORTSMULTICORE_UNSTABLE'='quiet' to 'logical' would result in missing values for option 'parallelly.supportsMulticore.unstable': 'NA'

This error makes the parallel worker to believe the future package is not installed (because it tests with require("future") which returns FALSE).

I've fixed this in parallelly (https://github.com/HenrikBengtsson/parallelly/commit/d908d462427ae08c73330e466a235d52e2608c76). A workaround until this is on CRAN, which will be a while, is to set the env var to false:

> Sys.setenv(R_FUTURE_SUPPORTSMULTICORE_UNSTABLE = "false")
> library(future)

PS. Please use the name R_PARALLELLY_SUPPORTSMULTICORE_UNSTABLE instead, since these features were moved to the parallelly package in October 2020. The R_FUTURE_* prefix is an alias for now.

[michalovadek]

I'm running into this error as well when executing plan(multisession) but without even changing Sys.setenv(R_FUTURE_SUPPORTSMULTICORE_UNSTABLE): Error: Initialization of plan() failed, because the test future used for validation failed. The reason was: Package 'future' is not installed on worker (r_version: 4.1.1 (2021-08-10); platform: x86_64-w64-mingw32 (64-bit); os: Windows 10 x64 build 19042

I have tried the workaround described here, as well as updating all relevant packages, including to devel versions. The error remains the same throughout.

R version 4.1.1 (2021-08-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale: [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252 [4] LC_NUMERIC=C LC_TIME=English_United Kingdom.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] future_1.23.0-9004

loaded via a namespace (and not attached): [1] compiler_4.1.1 parallelly_1.29.0 parallel_4.1.1 tools_4.1.1 listenv_0.8.0 codetools_0.2-18 [7] digest_0.6.29 globals_0.14.0

[HenrikBengtsson]

What does your .libPaths() looks like?

[michalovadek]

.libPaths() [1] "\\home.ser.ver.com/home-XO$/user/Documents/R/win-library/4.1" [2] "C:/Program Files/R/R-4.1.1/library"

[HenrikBengtsson]

@michalovadek, I suspect your problem is indepent of what was reported in this issue, which turned out to be a bug in **parallelly (< 1.28.0) that was triggered when using R_FUTURE_SUPPORTSMULTICORE_UNSTABLE = "quiet". I've created https://github.com/HenrikBengtsson/future/discussions/566 - let's continue the discussion there.