Q_rows fails with log_worker=F

[dleopold]

I am having trouble getting the Q_rows function to work when not writing the logs to file. Here is a toy example:

library(clustermq)
options(clustermq.scheduler="multiprocess")

fxn <- function(...){list(...)}

dat <- data.frame(A=1:4000,
                  B=1:4000)
# This fails:
out <- Q_rows(dat, fxn, n_jobs=parallel::detectCores(), log_worker = F)
# This works:
out <- Q_rows(dat, fxn, n_jobs=parallel::detectCores(), log_worker = T)

With a smaller data frame the example without logging works:

dat <- data.frame(A=1:100,
                  B=1:100)
out <- Q_rows(dat, fxn, n_jobs=parallel::detectCores(), log_worker = F)

but with warnings as mentioned in #269.

> warnings()
Warning messages:
1: In sprintf(log_file, i) : one argument not used by format '|'
2: In sprintf(log_file, i) : one argument not used by format '|'
3: In sprintf(log_file, i) : one argument not used by format '|'
4: In sprintf(log_file, i) : one argument not used by format '|'

[dleopold]

in case it helps, here is some on the R environment where I am seeing this behavior:

> sessionInfo()
R version 4.1.1 (2021-08-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Pop!_OS 20.04 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] clustermq_0.8.96

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7        codetools_0.2-18  prettyunits_1.1.1 ps_1.6.0          crayon_1.4.1      R6_2.5.1         
 [7] lifecycle_1.0.0   rlang_0.4.11      progress_1.2.2    callr_3.7.0       vctrs_0.3.8       ellipsis_0.3.2   
[13] tools_4.1.1       hms_1.1.0         parallel_4.1.1    compiler_4.1.1    processx_3.5.2    pkgconfig_2.0.3  

[mschubert]

Thanks, there was indeed an issue with the callr::r_bg default file "|" that somehow caused clustermq to hang.

I've now requested a base::nullfile explicitly, which seems to solve this problem.

Please let me know if that works for you too! (and reopen if it does not)