Error in makeClusterPSOCK() on CRAN MacOS environments

[idblr]

I am a maintainer for a package the Imports the future package and I have a CRAN RMD Error that was traced to parallelly::makeClusterPSOCK()

Error: Cluster setup failed. 2 of 2 workers failed to connect.

This error only appears on CRAN MacOS environments and throws no errors on other CRAN environments. Based on your note "a bug in R (>= 4.0.0 && <= 4.1.0) causing the new setup_strategy = "parallel" to fail in the RStudio Console on some systems" from parallely 1.26.1 release I tried adding:

skip_if_not_installed("parallelly", minimum_version = "1.26.1")

to my testthat script, but the error was thrown on the rhub macos server (CRAN configuration) used for checking.

If I add:

skip_on_os("mac")

to my testthat script the rhub macos server throws no error. When I check the package on my local MacOS environment (R version 4.1.0, parallelly version 1.26.1, future version 1.21.0) the checks are clean so I suspect this may be a CRAN issue. Any insight on this error would be much appreciated, thank you!

[HenrikBengtsson]

What's the package?

[idblr]

Hi, @HenrikBengtsson. The package is sparrpowR

GitHub CRAN Check

I am using future::multiprocess and see your post about better syntax that I am in the process of implementing for the next CRAN release

[HenrikBengtsson]

... I have a CRAN RMD Error that was traced to parallelly::makeClusterPSOCK()

Error: Cluster setup failed. 2 of 2 workers failed to connect.

How do you see that error message on https://cran.r-project.org/web/checks/check_results_sparrpowR.html? Basically, how do you know it's that error?

I am using future::multiprocess ...

Did you mean future::multisession?

[idblr]

@HenrikBengtsson Yes, I am using future::multisession (Apologies for the confusion, not future::multiprocess)

Unfortunately, the CRAN checks only show the first 13 lines. When I run the package on the rhub MacOS environment I can find some additional diagnostic information in the testhat.Rout.fail file within the artifacts.

I include the full error traceback here (after some textual clean-up):

Error (test-test-spatial_power.R:201:3): future package functions properly
Error: Cluster setup failed. 2 of 2 workers failed to connect.
Backtrace:

  1. testthat::expect_named(...) test-test-spatial_power.R:201:2
  2. testthat::quasi_label(enquo(object), label, arg = "object")
  3. rlang::eval_bare(expr, quo_get_env(quo))
  4. sparrpowR::spatial_power(...)
  5.   future::plan(future::multisession, workers = n_core)
  6.     future:::plan_set(...)
  7.       future:::plan_init()
  8.         future:::evaluator(...)
  9.           future:::strategy(..., envir = envir, workers = workers)
 10.             future::MultisessionFuture(...)
 11.               future::ClusterFuture(...)
 12.                 base::do.call(...)
 13.                 (function (future, workers = NULL, ...) ...
 14.                   future:::ClusterRegistry("start", workers = workers, ...)
 15.                     future:::makeCluster(workers, ...)
 16.                       parallelly::makeClusterPSOCK(workers, ...)

[HenrikBengtsson]

I've managed to reproduce this with a minimal package example, cf. https://github.com/HenrikBengtsson/teeny/runs/3076390940?check_suite_focus=true.

This is R bug 18119 kicking in because the tcltk package is loaded, which it is when your package is attached.

Workaround

I've updated parallelly (>= 1.26.1-9002) to detect tcltk and force setup_strategy = "sequential" for R (>= 4.0.0 && <= 4.1.0).

Until that version is on CRAN, add the following to your tests/testthat.R:

## WORKAROUND: Avoid R bug 18119 [1] that is trigger when for instance the
## 'tcltk' package is loaded on macOS, or when running in the RStudio Console
## [1] https://bugs.r-project.org/bugzilla/show_bug.cgi?id=18119
if (getRversion() >= "4.0.0" && getRversion() <= "4.1.0") {
  options(parallelly.makeNodePSOCK.setup_strategy = "sequential")
}

That should fix it for you. Let me know how it goes.

[idblr]

Thanks, @HenrikBengtsson. I have made the changes to my testhat and they passed checks. I have submitted to CRAN and will return if the error persists.

[HenrikBengtsson]

Did the CRAN team reach out to you asking to fix this as soon as possible, or did you notice the problem yourself?

[idblr]

@HenrikBengtsson, I noticed the problem on my own after v0.2.2 release binaries were built in June 2021. I have not heard from the CRAN team.

[idblr]

The fix seems to work for MacOS environments but now there is an error for Linux Debian and Fedora environments.

sparrpowR check results envi check results

This time I was contacted by the CRAN team.

[HenrikBengtsson]

Hi. Can you please open a new issue for this; this is most likely different from this issue. Also, since it's not clear what version of parallelly those CRAN checks ran on, can you reproduce this on R-Hub?