Suppose hpc is the headnode of a PBS-based cluster,
sos run -r hpc -q hpc
will cause sos to
-r hpc willl execute the entire workflow on hpc. Because hpc has a pbs queue, the entire workflow will be submitted as a single-node PBS job
-q hpc will let the workflow, now executed on a computing node, submit jobs from hpc
For step 2, this requires computing node can access the head node in order to submit jobs (ssh headnode qsub ....), which is at least not the case for the cluster system at our institution.
Now, assuming that hpc_headnode is defined as a regular queue (not pbs),
sos run -r hpc_headnode -q hpc
will
-r hpc_headnode will execute the workflow on the headnode. This means the sos process will remain active during the execution of the workflow, which is not allowed for many cluster systems.
-q hoc will submit the job from the headnode.
The problem with this approach is that, with the current implementation, the job will be submitted with ssh hpc ... from hpc_headnode. Public key authentication does not work for this case.
Suppose
hpcis the headnode of a PBS-based cluster,will cause sos to
-r hpcwilll execute the entire workflow on hpc. Because hpc has a pbs queue, the entire workflow will be submitted as a single-node PBS job-q hpcwill let the workflow, now executed on a computing node, submit jobs from hpcFor step 2, this requires computing node can access the head node in order to submit jobs (
ssh headnode qsub ....), which is at least not the case for the cluster system at our institution.Now, assuming that
hpc_headnodeis defined as a regular queue (notpbs),will
-r hpc_headnodewill execute the workflow on the headnode. This means thesosprocess will remain active during the execution of the workflow, which is not allowed for many cluster systems.-q hocwill submit the job from the headnode.The problem with this approach is that, with the current implementation, the job will be submitted with
ssh hpc ...fromhpc_headnode. Public key authentication does not work for this case.