search

HenrikBengtsson / future

:rocket: R package: future: Unified Parallel and Distributed Processing in R for Everyone
https://future.futureverse.org
946 stars 82 forks source link

plan(cluster) over ssh asks for port to be specified in workRSOCK() #694

Closed iagomosqueira closed 11 months ago

iagomosqueira commented 11 months ago

Describe the bug

Setting up a plan using 'cluster' to use multiple cores via ssh on a remote machine. R processes start but return an error about Error in workRSOCK() : PORT must be specified

> plan(cluster, workers=rep("USER@ip.of.my.server", 6), rscript='/home/WUR/mosqu003/Bin/R')

ARGUMENT 'MASTER=localhost' __ignored__
ARGUMENT 'PORT=11636' __ignored__
ARGUMENT 'OUT=/dev/null' __ignored__
ARGUMENT 'TIMEOUT=2592000' __ignored__
ARGUMENT 'XDR=FALSE' __ignored__
ARGUMENT 'SETUPTIMEOUT=120' __ignored__
ARGUMENT 'SETUPSTRATEGY=sequential' __ignored__

R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

[...]

> options(socketOptions = "no-delay")
> workRSOCK <- tryCatch(parallel:::.workRSOCK, error=function(e) parallel:::.slaveRSOCK); workRSOCK()
Error in workRSOCK() : PORT must be specified
Execution halted

and the plan hangs until killed.

Reproduce example

I cannot provide access to that machine, command issued as above

Expected behavior

Connection is established and future is available.

Session information

Please share your session information after the error has occurred so that we also see which packages and versions are involved;

> sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

time zone: Etc/UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
 [1] processx_3.8.2    compiler_4.3.1    R6_2.5.1          rprojroot_2.0.3
 [5] cli_3.6.1         prettyunits_1.1.1 tools_4.3.1       curl_5.0.1
 [9] crayon_1.5.2      remotes_2.4.2.1   desc_1.4.2        callr_3.7.3
[13] pkgbuild_1.4.2    ps_1.7.5
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C               LC_TIME=en_GB.UTF-8        LC_COLLATE=en_GB.UTF-8     LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8
 [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C

time zone: Europe/Amsterdam
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base

other attached packages:
 [1] mse_2.2.3.9183       progressr_0.13.0     doFuture_1.0.0       future_1.33.0        foreach_1.5.2        FLasher_0.7.1.9206   FLFishery_0.3.8.9008 ggplotFL_2.7.0.9126
 [9] ggplot2_3.4.2        FLCore_2.6.19.9066   iterators_1.0.14     lattice_0.21-8       nvimcom_0.9-146      icesTAF_4.2.0        data.table_1.14.8    rmarkdown_2.23
[17] devtools_2.4.5       usethis_2.2.2        colorout_1.2-2

loaded via a namespace (and not attached):
 [1] tidyselect_1.2.0    dplyr_1.1.2         fastmap_1.1.1       promises_1.2.0.1    digest_0.6.33       mime_0.12           lifecycle_1.0.3     ellipsis_0.3.2
 [9] processx_3.8.2      magrittr_2.0.3      compiler_4.3.1      rlang_1.1.1         tools_4.3.1         utf8_1.2.3          knitr_1.43          prettyunits_1.1.1
[17] htmlwidgets_1.6.2   pkgbuild_1.4.2      xml2_1.3.5          pkgload_1.3.2.1     miniUI_0.1.1.1      withr_2.5.0         purrr_1.0.1         grid_4.3.1
[25] stats4_4.3.1        fansi_1.0.4         roxygen2_7.2.3      urlchecker_1.0.1    profvis_0.3.8       xtable_1.8-4        colorspace_2.1-0    data.tree_1.0.0
[33] globals_0.16.2      scales_1.2.1        MASS_7.3-60         cli_3.6.1           crayon_1.5.2        generics_0.1.3      remotes_2.4.2.1     future.apply_1.11.0
[41] sessioninfo_1.2.2   cachem_1.0.8        stringr_1.5.0       parallel_4.3.1      vctrs_0.6.3         Matrix_1.6-0        callr_3.7.3         patchwork_1.1.2
[49] ggrepel_0.9.3       listenv_0.9.0       glue_1.6.2          parallelly_1.36.0   codetools_0.2-19    ps_1.7.5            cowplot_1.1.1       stringi_1.7.12
[57] gtable_0.3.3        later_1.3.1         FLBRP_2.5.9.9007    munsell_0.5.0       tibble_3.2.1        TAF_4.2.1           pillar_1.9.0        htmltools_0.5.5
[65] R6_2.5.1            evaluate_0.21       shiny_1.7.4.1       memoise_2.0.1       httpuv_1.6.11       Rcpp_1.0.11         gridExtra_2.3       xfun_0.39
[73] fs_1.6.3            pkgconfig_2.0.3
HenrikBengtsson commented 11 months ago

If you specify a custom rscript argument, it must point to the Rscript executable - not R.

iagomosqueira commented 11 months ago

Thanks very much, I should have guessed as much