potential conflict between doParallel / foreach and qgam

[thmschk]

I just noticed a strange behaviour when loading the package qgam.

The following MWE works as expected:

R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 20.1

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

attached base packages:
[1] parallel  stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
[1] doParallel_1.0.16 iterators_1.0.13  foreach_1.5.1     qgam_1.3.3        mgcv_1.8-36       nlme_3.1-152

####

library(doParallel)

no.cluster <- parallel::detectCores() - 2

cl <- parallel::makeForkCluster(no.cluster)
doParallel::registerDoParallel(cl)

foreach(i = seq(1,3,1)) %dopar% {

 say <- "hello"

}

However, when I load qgam, ich get the following error:

 library(qgam) 
 library(doParallel)

 no.cluster <- parallel::detectCores() - 2

 cl <- parallel::makeForkCluster(no.cluster)
 doParallel::registerDoParallel(cl)

 foreach(i = seq(1,3,1)) %dopar% {

  say <- "hello"

 }

> Error in serialize(data, node$con) : error writing to connection

It seems like, qgam opens any worker connection in the background

[mfasiolo]

Hi Lorenz,

Thanks for reporting this. I could not reproduce the error on my laptop (session info below). Could you includ the output of sessionInfo() when you get the error?

> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_GB.UTF-8        LC_COLLATE=en_GB.UTF-8    
 [5] LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8   
 [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] doParallel_1.0.15 iterators_1.0.12  foreach_1.5.0     qgam_1.3.2       
[5] mgcv_1.8-35       nlme_3.1-144     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.5       lattice_0.20-40  codetools_0.2-16 later_1.1.0.1   
 [5] digest_0.6.25    mime_0.9         grid_3.6.3       plyr_1.8.6      
 [9] R6_2.4.1         xtable_1.8-4     magrittr_1.5     rlang_0.4.7     
[13] promises_1.1.1   Matrix_1.2-18    splines_3.6.3    shiny_1.5.0     
[17] fastmap_1.0.1    httpuv_1.5.4     compiler_3.6.3   htmltools_0.5.0

[thmschk]

Hi Matteo,

thanks for taking care. Meanwhile I had to reinstall my packages, surprisingly the error still remains, but is different now.

> library(qgam)           
Lade nötiges Paket: mgcv
Lade nötiges Paket: nlme
This is mgcv 1.8-36. For overview type 'help("mgcv-package")'.
> library(doParallel)     
Lade nötiges Paket: foreach
Lade nötiges Paket: iterators
Lade nötiges Paket: parallel
> no.cluster <- parallel::detectCores() - 2
>  cl <- parallel::makeForkCluster(no.cluster)
> doParallel::registerDoParallel(cl)
> foreach(i = seq(1,3,1)) %dopar% {
+   
+   say <- "hello"
+   
+  }
Fehler in serialize(data, node$con, xdr = FALSE) : ignoring SIGPIPE signal
> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 20.1

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=de_DE.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=de_DE.UTF-8        LC_COLLATE=de_DE.UTF-8    
 [5] LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=de_DE.UTF-8   
 [7] LC_PAPER=de_DE.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] doParallel_1.0.16 iterators_1.0.13  foreach_1.5.1     qgam_1.3.3       
[5] mgcv_1.8-36       nlme_3.1-152     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.6        magrittr_2.0.1    splines_4.1.0     xtable_1.8-4     
 [5] lattice_0.20-44   R6_2.5.0          rlang_0.4.11      fastmap_1.1.0    
 [9] plyr_1.8.6        tools_4.1.0       grid_4.1.0        htmltools_0.5.1.1
[13] ellipsis_0.3.2    digest_0.6.27     lifecycle_1.0.0   Matrix_1.3-4     
[17] shiny_1.6.0       later_1.2.0       promises_1.2.0.1  codetools_0.2-18 
[21] mime_0.11         compiler_4.1.0    httpuv_1.6.1     
> 

[mfasiolo]

Hi Lorenz,

Sorry for the late reply, I am on holiday. The main thing that qgam does is to load mgcv and to import the doParallel and parallel packages. Do you get the same error if you substitute library(qgam) with library(mgcv) or with library(parallel)?