HenrikBengtsson
commented
4 years ago Hi. It looks like qstat is not available on that machine; verify by calling qstat on the shell command line. Second, this looks like an issue with configuring batchtools rather than an issue with the future framework - I'd ask for advice over there. Third, many of your packages are out of date, so I'd update all of them too, because it's unlikely that someone will be willing to troubleshooting further without you running the latest versions. Fourth, your R version is also very much out of date too - if you can, get that updated (but I don't think that's the problem here).
nick-youngblut
Error
``` Error in OSError("Listing of jobs failed", res) : Listing of jobs failed (exit code 127); cmd: 'qstat -u $USER -s rs' output: command not found ```Code that I'm running
``` library(future) library(future.batchtools) options(future.wait.interval = 2.0) resources = list(h_rt = '00:59:00', h_vmem = '5G', threads = '1', conda.env = 'py3_physeq_test') plan(batchtools_sge, resources=resources, workers=20) # set functions (packages added with `%packages%`) x %<-% { Sys.sleep(2); 3.14 } %packages% c("data.table", "ggplot2") y %<-% { Sys.sleep(2); 2.71 } %packages% c("data.table", "ggplot2") z %<-% { Sys.sleep(2); 10.71 } %packages% c("data.table", "ggplot2") # conduct calculation x + y + z ```~/.batchtools.tmpl
``` #!/bin/bash ## The name of the job, can be anything, simply used when displaying the list of running jobs #$ -N <%= job.name %> ## Combining output/error messages into one file #$ -j y ## Giving the name of the output log file #$ -o <%= log.file %> ## One needs to tell the queue system to use the current directory as the working directory ## Or else the script may fail as it will execute in your top level home directory /home/username #$ -cwd ## Use environment variables #$ -V ## Number of threads #$ -pe parallel <%= resources$threads %> ## time #$ -l h_rt=<%= resources$h_rt %> ## memory #$ -l h_vmem=<%= resources$h_vmem %> . ~/.bashrc conda activate <%= resources$conda.env %> ## Export value of DEBUGME environemnt var to slave export DEBUGME=<%= Sys.getenv("DEBUGME") %> <%= sprintf("export OMP_NUM_THREADS=%i", resources$omp.threads) -%> <%= sprintf("export OPENBLAS_NUM_THREADS=%i", resources$blas.threads) -%> <%= sprintf("export MKL_NUM_THREADS=%i", resources$blas.threads) -%> Rscript -e 'batchtools::doJobCollection("<%= uri %>")' exit 0 ```~/.batchtools.conf.R
``` default.resources = list(h_rt = '00:59:00', h_vmem = '4G', threads = '1', conda.env = "py3") cluster.functions = makeClusterFunctionsSGE(template = "~/.batchtools.tmpl") temp.dir = "/ebio/abt3_projects/temp_data/" ```sessionInfo
``` R version 3.5.1 (2018-07-02) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 18.04.3 LTS Matrix products: default BLAS: /opt/microsoft/ropen/3.5.1/lib64/R/lib/libRblas.so LAPACK: /opt/microsoft/ropen/3.5.1/lib64/R/lib/libRlapack.so locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] future.apply_1.0.0 future.batchtools_0.7.1 future_1.9.0 [4] dplyr_0.7.6 RevoUtils_11.0.1 RevoUtilsMath_11.0.0 loaded via a namespace (and not attached): [1] Rcpp_0.12.18 pillar_1.3.0 compiler_3.5.1 bindr_0.1.1 [5] prettyunits_1.0.2 progress_1.2.0 base64enc_0.1-3 tools_3.5.1 [9] digest_0.6.15 packrat_0.4.9-3 jsonlite_1.5 evaluate_0.11 [13] tibble_1.4.2 checkmate_1.8.5 pkgconfig_2.0.1 rlang_0.2.1 [17] yaml_2.2.0 parallel_3.5.1 bindrcpp_0.2.2 withr_2.1.2 [21] stringr_1.3.1 knitr_1.20 fs_1.2.5 rappdirs_0.3.1 [25] hms_0.4.2 globals_0.12.1 rprojroot_1.3-2 tidyselect_0.2.4 [29] glue_1.3.0 data.table_1.11.4 listenv_0.7.0 R6_2.2.2 [33] base64url_1.4 rmarkdown_1.10 purrr_0.2.5 magrittr_1.5 [37] backports_1.1.2 codetools_0.2-15 batchtools_0.9.10 htmltools_0.3.6 [41] assertthat_0.2.0 brew_1.0-6 stringi_1.2.4 crayon_1.3.4 ```