search

TACC / launcher

A simple utility for executing multiple sequential or multi-threaded applications in a single multi-node batch job
MIT License
63 stars 33 forks source link

More jobs than processes #68

Open dyhan316 opened 1 year ago

dyhan316 commented 1 year ago

Hi! I'm using launcher in stampede 2, but as the picture below shows, only 44 processes are initialized, even when I run total jobs to be 45. As a result, one of the 45 jobs does not run properly...

image

login2.stampede2(1053)$ cat skx_normal_QSIPREP_preproc.o10571265 | grep 43
Launcher: Task 43 running job 4 on c457-228.stampede2.tacc.utexas.edu (/work2/08834/tg881334/stampede2/ABCD_preproc/IMP_ABCD_preproc/step1_shell_outputs/run_this/run_shell_sub-NDARINV0YE7L9KU.sh >> /work2/08834/tg881334/stampede2/ABCD_preproc/IMP_ABCD_preproc/step1_shell_outputs/log_outputs/output-${LAUNCHER_TSK_ID}_sub-NDARINV0YE7L9KU)
Launcher: Task 43 running job 13 on c457-228.stampede2.tacc.utexas.edu (/work2/08834/tg881334/stampede2/ABCD_preproc/IMP_ABCD_preproc/step1_shell_outputs/run_this/run_shell_sub-NDARINV01ELX9L6.sh >> /work2/08834/tg881334/stampede2/ABCD_preproc/IMP_ABCD_preproc/step1_shell_outputs/log_outputs/output-${LAUNCHER_TSK_ID}_sub-NDARINV01ELX9L6)

as the output log shows, there's two task 43's, which I believe is the reason for this bug. Is there a way to fix this?