mllg / batchtools

Tools for computation on batch systems
https://mllg.github.io/batchtools/
GNU Lesser General Public License v3.0
172 stars 51 forks source link

Error in stop(simpleError(sprintf(...), call = sys.call(sys.parent()))) #248

Closed nick-youngblut closed 4 years ago

nick-youngblut commented 4 years ago
Error & traceback ``` Error in stop(simpleError(sprintf(...), call = sys.call(sys.parent()))) : bad error message 9. stop(simpleError(sprintf(...), call = sys.call(sys.parent()))) 8. stopf("%s (exit code %i);\ncmd: '%s'\noutput:\n%s", msg, res$exit.code, stri_flatten(c(res$sys.cmd, res$sys.args), collapse = " "), stri_flatten(res$output, "\n")) 7. OSError("Listing of jobs failed", res) 6. listJobs(reg, c(quote("-u $USER"), "-s rs")) 5. cf$listJobsRunning(reg) 4. unique(cf$listJobsRunning(reg)) 3. getBatchIds(reg, status = status) 2. .findOnSystem(reg = reg, cols = c("job.id", "batch.id")) 1. submitJobs(resources = list(h_vmem = "2G", threads = "1")) ```
Code that I'm running ``` require(batchtools) test.batch <- function(ele) { t = sample(20:50, 1) print(paste0("waking up after ", t, " seconds.")) print(paste0("of course you sent me: ", ele, " bucks.")) } reg = makeRegistry(conf.file = "~/.batchtools.conf.R") batchMap(fun = test.batch, ele = sample(10), reg=reg) submitJobs(resources = list(h_vmem = "2G", threads = "1")) ```
`tree` on the registry directory ``` registry/ ├── exports ├── external ├── jobs ├── logs ├── registry.rds ├── results ├── updates └── user.function.rds ```
~/.batchtools.tmpl ``` #!/bin/bash ## The name of the job, can be anything, simply used when displaying the list of running jobs #$ -N <%= job.name %> ## Combining output/error messages into one file #$ -j y ## Giving the name of the output log file #$ -o <%= log.file %> ## One needs to tell the queue system to use the current directory as the working directory ## Or else the script may fail as it will execute in your top level home directory /home/username #$ -cwd ## Use environment variables #$ -V ## Number of threads #$ -pe parallel <%= resources$threads %> ## time #$ -l h_rt=<%= resources$h_rt %> ## memory #$ -l h_vmem=<%= resources$h_vmem %> . ~/.bashrc conda activate <%= resources$conda.env %> ## Export value of DEBUGME environemnt var to slave export DEBUGME=<%= Sys.getenv("DEBUGME") %> <%= sprintf("export OMP_NUM_THREADS=%i", resources$omp.threads) -%> <%= sprintf("export OPENBLAS_NUM_THREADS=%i", resources$blas.threads) -%> <%= sprintf("export MKL_NUM_THREADS=%i", resources$blas.threads) -%> Rscript -e 'batchtools::doJobCollection("<%= uri %>")' exit 0 ```
~/.batchtools.conf.R ``` default.resources = list(h_rt = '00:59:00', h_vmem = '4G', threads = '1', conda.env = "py3") cluster.functions = makeClusterFunctionsSGE(template = "~/.batchtools.tmpl") temp.dir = "/ebio/abt3_projects/temp_data/" ```
sessionInfo ``` R version 3.5.1 (2018-07-02) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 18.04.3 LTS Matrix products: default BLAS: /opt/microsoft/ropen/3.5.1/lib64/R/lib/libRblas.so LAPACK: /opt/microsoft/ropen/3.5.1/lib64/R/lib/libRlapack.so locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] batchtools_0.9.10 data.table_1.11.4 RevoUtils_11.0.1 RevoUtilsMath_11.0.0 loaded via a namespace (and not attached): [1] Rcpp_0.12.18 knitr_1.20 magrittr_1.5 hms_0.4.2 [5] progress_1.2.0 rappdirs_0.3.1 R6_2.2.2 brew_1.0-6 [9] rlang_0.2.1 stringr_1.3.1 tools_3.5.1 packrat_0.4.9-3 [13] checkmate_1.8.5 withr_2.1.2 htmltools_0.3.6 base64url_1.4 [17] yaml_2.2.0 digest_0.6.15 assertthat_0.2.0 rprojroot_1.3-2 [21] crayon_1.3.4 fs_1.2.5 evaluate_0.11 rmarkdown_1.10 [25] stringi_1.2.4 compiler_3.5.1 backports_1.1.2 prettyunits_1.0.2 [29] pkgconfig_2.0.1 ```
mllg commented 4 years ago

Thanks for the report. I've tried to make the error message helper more robust, but cannot test it without a SGE system. Can you try if the latest version solves your problem?

mllg commented 4 years ago

Closed due to inactivity. Re-open if the problem persists.