problem with future: plan() [Error: ‘node$session_info$process$pid == pid’ is not TRUE]

[AIYang1210]

Hi， I run plan(), but it reported errors. Please help me to solve this. Thanks! Windows 10 R version 4.2.0 (2022-04-22 ucrt)

library(future) plan(multisession, workers=5) Error: ‘node$session_info$process$pid == pid’ is not TRUE In addition: Warning message: In add_cluster_session_info(cl[ii]) : NAs introduced by coercion

nbrOfWorkers() [1] 5

plan(multisession, workers=4) Error: ‘node$session_info$process$pid == pid’ is not TRUE In addition: Warning message: In add_cluster_session_info(cl[ii]) : NAs introduced by coercion nbrOfWorkers() [1] 4

availableCores() system 8 plan(multisession) Error: ‘node$session_info$process$pid == pid’ is not TRUE In addition: Warning message: In add_cluster_session_info(cl[ii]) : NAs introduced by coercion

[HenrikBengtsson]

Hi, thanks for reporting. This shouldn't happen. Let's investigate. What do the following three code-snippets output on your machine:

cl <- parallelly::makeClusterPSOCK(5)
void <- lapply(cl, print)

and what does:

Sys.getlocale()

output, and finally,

sessionInfo()

?

[AIYang1210]

Hi HenrikBengtsson! Thank you very much for your reply!

Interesting, when I ran future::plan("multicore", workers = 4), it worked well.

Finally, I solved this problem. remove(list = ls()) and restart the session twice.
Now, future::plan("multisession", workers = 4) , worked well.

Also, I attach the results you asked for. Please see below. Thanks!

remove(list = ls()) library(future) cl <- parallelly::makeClusterPSOCK(5) void <- lapply(cl, print) node of a socket cluster on host ‘localhost’ with pid 33172 node of a socket cluster on host ‘localhost’ with pid 35348 node of a socket cluster on host ‘localhost’ with pid 32348 node of a socket cluster on host ‘localhost’ with pid 7476 node of a socket cluster on host ‘localhost’ with pid 25888 Sys.getlocale() [1] "LC_COLLATE=Chinese (Simplified)_China.utf8;LC_CTYPE=Chinese (Simplified)_China.utf8;LC_MONETARY=Chinese (Simplified)_China.utf8;LC_NUMERIC=C;LC_TIME=Chinese (Simplified)_China.utf8" sessionInfo() R version 4.2.0 (2022-04-22 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale: [1] LC_COLLATE=Chinese (Simplified)_China.utf8 LC_CTYPE=Chinese (Simplified)_China.utf8 LC_MONETARY=Chinese (Simplified)_China.utf8 [4] LC_NUMERIC=C LC_TIME=Chinese (Simplified)_China.utf8

attached base packages: [1] parallel stats graphics grDevices utils datasets methods base

other attached packages: [1] umap_0.2.8.0 doParallel_1.0.17 iterators_1.0.14 foreach_1.5.2 svglite_2.1.0 ggalluvial_0.12.3 NMF_0.24.0
[8] cluster_2.1.3 rngtools_1.5.2 pkgmaker_0.32.2 registry_0.5-1 forcats_0.5.1 stringr_1.4.0 purrr_0.3.4
[15] readr_2.1.2 tidyr_1.2.0 tibble_3.1.6 tidyverse_1.3.1 SeuratData_0.2.2 sp_1.4-7 SeuratObject_4.1.0 [22] Seurat_4.1.1 reticulate_1.24 tensorflow_2.8.0 patchwork_1.1.1 CellChat_1.4.0 Biobase_2.56.0 BiocGenerics_0.42.0 [29] ggplot2_3.3.6 igraph_1.3.1 dplyr_1.0.9 devtools_2.4.3 usethis_2.1.5 future_1.25.0

loaded via a namespace (and not attached): [1] utf8_1.2.2 tidyselect_1.1.2 htmlwidgets_1.5.4 grid_4.2.0 Rtsne_0.16 munsell_0.5.0 codetools_0.2-18
[8] ica_1.0-2 miniUI_0.1.1.1 withr_2.5.0 spatstat.random_2.2-0 colorspace_2.0-3 progressr_0.10.0 highr_0.9
[15] knitr_1.39 rstudioapi_0.13 stats4_4.2.0 ROCR_1.0-11 tensor_1.5 listenv_0.8.0 labeling_0.4.2
[22] polyclip_1.10-0 farver_2.1.0 rprojroot_2.0.3 coda_0.19-4 parallelly_1.31.1 vctrs_0.4.1 generics_0.1.2
[29] xfun_0.31 R6_2.5.1 clue_0.3-60 spatstat.utils_2.3-1 cachem_1.0.6 assertthat_0.2.1 promises_1.2.0.1
[36] scales_1.2.0 rgeos_0.5-9 gtable_0.3.0 globals_0.15.0 processx_3.5.3 goftest_1.2-3 rlang_1.0.2
[43] systemfonts_1.0.4 GlobalOptions_0.1.2 splines_4.2.0 lazyeval_0.2.2 broom_0.8.0 spatstat.geom_2.4-0 modelr_0.1.8
[50] BiocManager_1.30.17 reshape2_1.4.4 abind_1.4-5 backports_1.4.1 httpuv_1.6.5 tools_4.2.0 gridBase_0.4-7
[57] statnet.common_4.6.0 ellipsis_0.3.2 spatstat.core_2.4-2 RColorBrewer_1.1-3 sessioninfo_1.2.2 ggridges_0.5.3 Rcpp_1.0.8.3
[64] plyr_1.8.7 base64enc_0.1-3 ps_1.7.0 prettyunits_1.1.1 openssl_2.0.1 rpart_4.1.16 deldir_1.0-6
[71] pbapply_1.5-0 GetoptLong_1.0.5 cowplot_1.1.1 S4Vectors_0.34.0 zoo_1.8-10 haven_2.5.0 ggrepel_0.9.1
[78] here_1.0.1 fs_1.5.2 magrittr_2.0.3 data.table_1.14.2 RSpectra_0.16-1 sna_2.6 scattermore_0.8
[85] circlize_0.4.15 reprex_2.0.1 lmtest_0.9-40 RANN_2.6.1 whisker_0.4 fitdistrplus_1.1-8 matrixStats_0.62.0
[92] pkgload_1.2.4 hms_1.1.1 mime_0.12 evaluate_0.15 xtable_1.8-4 readxl_1.4.0 IRanges_2.30.0
[99] gridExtra_2.3 shape_1.4.6 tfruns_1.5.0 testthat_3.1.4 compiler_4.2.0 KernSmooth_2.23-20 crayon_1.5.1
[106] htmltools_0.5.2 tzdb_0.3.0 mgcv_1.8-40 later_1.3.0 lubridate_1.8.0 DBI_1.1.2 dbplyr_2.1.1
[113] ComplexHeatmap_2.13.1 rappdirs_0.3.3 MASS_7.3-57 Matrix_1.4-1 brio_1.1.3 cli_3.3.0 pkgconfig_2.0.3
[120] plotly_4.10.0 spatstat.sparse_2.1-1 xml2_1.3.3 rvest_1.0.2 callr_3.7.0 digest_0.6.29 sctransform_0.3.3
[127] RcppAnnoy_0.0.19 spatstat.data_2.2-0 cellranger_1.1.0 leiden_0.4.2 uwot_0.1.11 curl_4.3.2 shiny_1.7.1
[134] rjson_0.2.21 nlme_3.1-157 lifecycle_1.0.1 jsonlite_1.8.0 network_1.17.1 askpass_1.1 desc_1.4.1
[141] viridisLite_0.4.0 fansi_1.0.3 pillar_1.7.0 lattice_0.20-45 fastmap_1.1.0 httr_1.4.3 pkgbuild_1.3.1
[148] survival_3.3-1 glue_1.6.2 remotes_2.4.2 FNN_1.1.3 png_0.1-7 stringi_1.7.6 memoise_2.0.1
[155] irlba_2.3.5 future.apply_1.9.0

[AIYang1210]

BTW, error occurred when I used cellchat. I am not sure maybe there is a conflict between future and cellchat.

[HenrikBengtsson]

Interesting, when I ran future::plan("multicore", workers = 4), it worked well.

That actually expected; it uses a completely different set of code.

Finally, I solved this problem. remove(list = ls()) and restart the session twice. Now, future::plan("multisession", workers = 4) , worked well.

I suspect something happened when you restarted R; what variables you had in your workspace should not matter.

Also, I attach the results you asked for. Please see below. Thanks!

Thanks. I now know what happens. You run with the zn_CN locale/language settings, and there's a bug in parallelly that makes some incorrect assumptions, resulting in the error.

Reproducible example

I can reproduce it as on Ubuntu 20.04 with Simplified Chinese installed:

> Sys.setLanguage("zh_CN")
> library(future)
> plan(multisession, workers = 2)
错误: ‘node$session_info$process$pid == pid’ is not TRUE
此外: Warning message:
In add_cluster_session_info(cl[ii]) : 强制改变过程中产生了NA
> 

and narrowing in on the parallelly package alone:

> Sys.setLanguage("zh_CN")
> cl <- parallelly::makeClusterPSOCK(1)
错误: ‘node$session_info$process$pid == pid’ is not TRUE
此外: Warning message:
In add_cluster_session_info(cl[ii]) : 强制改变过程中产生了NA
> 

Workaround

Without updating parallelly, one can work around this problem as:

options(parallelly.makeNodePSOCK.validate = (packageVersion("parallelly") > "1.31.1"))

Proof:

> Sys.setLanguage("zh_CN")
> plan(multisession, workers = 2)
> 

Solved

I've fixed in the next version of parallelly (https://github.com/HenrikBengtsson/parallelly/commit/2bc73a7a0e5a86505cca019d00340bbd381bf3da). Until that is available on CRAN, you can preinstall the develop version (https://parallelly.futureverse.org/#pre-release-version).

[AIYang1210]

Thank you, HenrikBengtsson. Yes, your solution works well! great job!

[HenrikBengtsson]

Good to hear. Thanks for reporting back.