mschubert / clustermq

R package to send function calls as jobs on LSF, SGE, Slurm, PBS/Torque, or each via SSH
https://mschubert.github.io/clustermq/
Apache License 2.0
146 stars 27 forks source link

Warning in sprintf(log_file, i): one argument not used by format '|' #269

Closed wlandau closed 3 years ago

wlandau commented 3 years ago

Recently, I started noticing a warning while using the multiprocess scheduler.

options(clustermq.scheduler = "multiprocess")
clustermq::Q(x = 1:2, fun = function(x) x, n_jobs = 2)
#> Starting 2 processes ...
#> Warning in sprintf(log_file, i): one argument not used by format '|'

#> Warning in sprintf(log_file, i): one argument not used by format '|'
#> Running 2 calculations (0 objs/0 Mb common; 1 calls/chunk) ...
#> Master: [0.7s 14.7% CPU]; Worker: [avg 57.0% CPU, max 2593083.0 Mb]
#> [[1]]
#> [1] 1
#> 
#> [[2]]
#> [1] 2

Created on 2021-07-19 by the reprex package (v2.0.0)

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.1.0 (2021-05-18) #> os macOS Catalina 10.15.7 #> system x86_64, darwin17.0 #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz America/New_York #> date 2021-07-19 #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date lib source #> backports 1.2.1 2020-12-09 [1] CRAN (R 4.1.0) #> callr 3.7.0 2021-04-20 [1] CRAN (R 4.1.0) #> cli 3.0.0 2021-06-30 [1] CRAN (R 4.1.0) #> clustermq 0.8.95.1 2020-07-13 [1] CRAN (R 4.1.0) #> codetools 0.2-18 2020-11-04 [1] CRAN (R 4.1.0) #> crayon 1.4.1 2021-02-08 [1] CRAN (R 4.1.0) #> digest 0.6.27 2020-10-24 [1] CRAN (R 4.1.0) #> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.1.0) #> evaluate 0.14 2019-05-28 [1] CRAN (R 4.1.0) #> fansi 0.5.0 2021-05-25 [1] CRAN (R 4.1.0) #> fs 1.5.0 2020-07-31 [1] CRAN (R 4.1.0) #> glue 1.4.2 2020-08-27 [1] CRAN (R 4.1.0) #> highr 0.9 2021-04-16 [1] CRAN (R 4.1.0) #> hms 1.1.0 2021-05-17 [1] CRAN (R 4.1.0) #> htmltools 0.5.1.1 2021-01-22 [1] CRAN (R 4.1.0) #> knitr 1.33 2021-04-24 [1] CRAN (R 4.1.0) #> lifecycle 1.0.0 2021-02-15 [1] CRAN (R 4.1.0) #> magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.1.0) #> pillar 1.6.1 2021-05-16 [1] CRAN (R 4.1.0) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.1.0) #> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.1.0) #> processx 3.5.2 2021-04-30 [1] CRAN (R 4.1.0) #> progress 1.2.2 2019-05-16 [1] CRAN (R 4.1.0) #> ps 1.6.0 2021-02-28 [1] CRAN (R 4.1.0) #> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.1.0) #> R6 2.5.0 2020-10-28 [1] CRAN (R 4.1.0) #> Rcpp 1.0.7 2021-07-07 [1] CRAN (R 4.1.0) #> reprex 2.0.0 2021-04-02 [1] CRAN (R 4.1.0) #> rlang 0.4.11 2021-04-30 [1] CRAN (R 4.1.0) #> rmarkdown 2.9 2021-06-15 [1] CRAN (R 4.1.0) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.1.0) #> stringi 1.7.3 2021-07-16 [1] CRAN (R 4.1.0) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.1.0) #> styler 1.5.1 2021-07-13 [1] CRAN (R 4.1.0) #> tibble 3.1.2 2021-05-16 [1] CRAN (R 4.1.0) #> utf8 1.2.1 2021-03-12 [1] CRAN (R 4.1.0) #> vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.1.0) #> withr 2.4.2 2021-04-18 [1] CRAN (R 4.1.0) #> xfun 0.24 2021-06-15 [1] CRAN (R 4.1.0) #> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.1.0) #> #> [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library ```

This could be because I upgraded my version of R. On R 4.1.0:

sprintf("pattern", "value")
#> Warning in sprintf("pattern", "value"): one argument not used by format
#> 'pattern'
#> [1] "pattern"

Created on 2021-07-19 by the reprex package (v2.0.0)

On R 4.0.3:

sprintf("pattern", "value")
#> [1] "pattern"

Created on 2021-07-19 by the reprex package (v0.3.0)

mschubert commented 3 years ago

Thanks, the warning should be harmless (but you may want to add a %i somewhere in your log file name to separate the different workers)

Proposed change: wrap qsys_multicore.r#L23 in suppressWarnings() (maybe better: catch warning and warn instead that worker logs will be mangled)

wlandau commented 3 years ago

Catching sprintf() warnings makes sense to me. Would it also be possible to configure the defaults so no warning arises when the user does not set a log?

mschubert commented 3 years ago

That's already the case (or at least it should be) because we're only trying to match the pattern if log_file is a character (default is NULL).

Are you sure you haven't set it in options()?

That's indeed a bug in multiprocess