search

mschubert / clustermq

R package to send function calls as jobs on LSF, SGE, Slurm, PBS/Torque, or each via SSH
https://mschubert.github.io/clustermq/
Apache License 2.0
145 stars 26 forks source link

common data warning message #287

Closed wlandau closed 1 year ago

wlandau commented 2 years ago

When common data is large, I see a message like:

Warning in (function (...)  :
  Common data is 42.8 Mb. Recommended limit is  (set by clustermq.data.warning option)
Submitting 200 worker jobs (ID: cmq6272) ...
Running 1,200 calculations (13 objs/42.8 Mb common; 1 calls/chunk) ...

where the recommended limit is missing if the clustermq.data.warning option is not set. Would you be open to writing getOption("clustermq.data.warning", 1000) in this line?

https://github.com/mschubert/clustermq/blob/78125771cda27373bf19eb0f38ac5106524d4eae/R/qsys.r#L82

Also, I'm a bit surprised that 42.8 Mb generated the warning if the default limit is 1000 Mb.