UCSF-Costello-Lab / LG3_Pipeline

The original LG3 pipeline
https://github.com/UCSF-Costello-Lab/LG3_Pipeline
0 stars 0 forks source link

C4: qsub wrapper does not support CLI option `-d <path>` #163

Closed HenrikBengtsson closed 2 years ago

HenrikBengtsson commented 2 years ago

So, I forgot that we have a qsub tool on C4 that emulates Torque/PBS qsub. However, it does not support CLI option -d <path> which set the working directory. The LG3 Pipeline uses it solely(*) as qsub -d ${PWD:?} ... (see below).

The reason for this is that Torque/PBS on TIPCC will launch jobs in the $HOME folder of the user unless -d <path> is specified. In contrast, qsub on C4 will launch jobs in the $PWD working directory for all jobs (https://unix.stackexchange.com/a/207399/124078). So, if CLI option -d <path> is the only thing preventing us from using the existing code as is on C4, we can probably work around it. I'm thinking of something like:

## TIPCC: On TIPCC, qsub will run the job in $HOME, unless otherwise specified
[[ ${CLUSTER} == "tipcc" ]] && QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"

(*) It also uses it as qsub -d "$PBS_O_WORKDIR" ..., but that should correspond to qsub -d ${PWD:?} ....

[henrik@c4-dev1 LG3_Pipeline]$ grep -E '[-]d ' -- runs_demo/_run_* *.pbs scripts/*.sh
runs_demo/_run_Align_gz:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Align_mem:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Align_no_trim:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Germline:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Merge:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Merge_QC:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Pindel:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_PostMut:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_PSCN:qsub -d "${PSCN_HOME}" -M "${EMAIL}" -m ae "${LG3_HOME}/PSCN.submit_all.pbs"
runs_demo/_run_Recal:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Recal_pass2:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Recal_step:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
runs_demo/_run_Trim:##QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
PSCN.submit_all.pbs:jid=$(qsub -d "$PBS_O_WORKDIR" -M "${EMAIL}" -m ae                          1.mpileup.pbs)
PSCN.submit_all.pbs:jid=$(qsub -d "$PBS_O_WORKDIR" -M "${EMAIL}" -m ae -W depend="afterok:$jid" 2.sequenza.pbs)
PSCN.submit_all.pbs:jid=$(qsub -d "$PBS_O_WORKDIR" -M "${EMAIL}" -m ae -W depend="afterok:$jid" 3.pscbs.pbs)
PSCN.submit_all.pbs:jid=$(qsub -d "$PBS_O_WORKDIR" -M "${EMAIL}" -m ae -W depend="afterok:$jid" 4.reports.pbs)
scripts/mutdet_submit.sh:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"
scripts/mutect2_submit.sh:QSUB_OPTS="${QSUB_OPTS} -d ${PWD:?}"

(Manually curated to not show other CLI options named -d)

HenrikBengtsson commented 2 years ago

Now qsub CLI option -d <path> is only set when supported, cf. https://github.com/UCSF-Costello-Lab/LG3_Pipeline/commit/346c734e91160f794b7706dff939977639b13b64.

Turns out that this is sufficient to use C4's qsub wrapper for Slurm.