"-P projectID" argument for PBS pro

[cgjosephlee]

Describe the bug My HPC is managed using PBS pro, and requiring an argument -P projectID to configure a job. I specified in run.cfg

cluster_options = -P projectID -W group_list=projectID -q QUEUE

But it seems that DRMAA do not support -P for PBS pro.

There are many discussions about difficulties of configuring DRMAA, and many dependencies are no longer maintained. Would you consider to move to a modern workflow manager (snakemake, nextflow, etc.)?

ps. libdrmaa is installed following this.

Error message

...
[INFO] 2021-03-08 13:37:11,851 Total jobs: 1
Traceback (most recent call last):
  File "/home/hsinhan97/.local/NextDenovo/nextDenovo", line 856, in <module>
    main(args)
  File "/home/hsinhan97/.local/NextDenovo/nextDenovo", line 598, in main
    task.run.start()
  File "/home/u0/hsinhan97/.local/NextDenovo/lib/task_control.py", line 187, in start
    self._sge()
  File "/home/u0/hsinhan97/.local/NextDenovo/lib/task_control.py", line 241, in _sge
    jobid = self.drmaa.runJob(jt)
  File "/home/hsinhan97/miniconda3/envs/NextDenovo_ENV/lib/python3.8/site-packages/drmaa/session.py", line 314, in runJob
    c(drmaa_run_job, jid, sizeof(jid), jobTemplate)
  File "/home/hsinhan97/miniconda3/envs/NextDenovo_ENV/lib/python3.8/site-packages/drmaa/helpers.py", line 302, in c
    return f(*(args + (error_buffer, sizeof(error_buffer))))
  File "/home/hsinhan97/miniconda3/envs/NextDenovo_ENV/lib/python3.8/site-packages/drmaa/errors.py", line 151, in error_check
    raise _ERRORS[code - 1](error_string)
drmaa.errors.InvalidAttributeValueException: code 14: Invalid native specification: -P projectID -W group_list=projectID -q QUEUE (Unsupported option: -P)

Genome characteristics

Input data

Config file

[General]
job_type = pbs # local, slurm, sge, pbs, lsf
job_prefix = nextDenovo
task = all # all, correct, assemble
rewrite = yes # yes/no
deltmp = yes
parallel_jobs = 100 # number of tasks used to run in parallel
input_type = raw # raw, corrected
read_type = ont # clr, ont, hifi
input_fofn = input.fofn
workdir = ./
# usetempdir = /tmp
cluster_options = -P projectID -W group_list=projectID -q QUEUE

[correct_option]
read_cutoff = 1k
genome_size = 400m # estimated genome size
pa_correction = 100 # number of corrected tasks used to run in parallel, each corrected task requires ~TOTAL_INPUT_BASES/4 bytes of memory usage.
sort_options = -m 40g -t 10
minimap2_options_raw = -t 10
correction_options = -p 10

[assemble_option]
minimap2_options_cns = -t 10
minimap2_options_map = -t 10
nextgraph_options = -a 1

Operating system

LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: RedHatEnterpriseServer
Description:    Red Hat Enterprise Linux Server release 7.3 (Maipo)
Release:        7.3
Codename:       Maipo

GCC

Python python 3.8.8

NextDenovo 2.4.0

To Reproduce (Optional)

Additional context (Optional)

[moold]

Actually, I can not find the -P option in PBS qsub Manual Page, so you can not set it in drmaa, because it does not implement this option.

[cgjosephlee]

That one is a really old version of qsub manual, and it is why I said drmaa is outdated. Refer to this PBS Pro manual v14.2 released in 2017:

ps. After some surveys, I think -P option already existed in 2013.

[moold]

OK, got it and thank you for your suggestions.

[moold]

Try the latest version