Closed effigies closed 3 years ago
Launcher seems to avoid #559 by requesting a single job with multiple nodes/cpus, and then doing it own "scheduling" and distribution of runs across those nodes available. Obviously suboptimal from the point of view of HPC configuration, BUT allows right away to submit those 1000 subjects to be processed.
so a prototypical invocation would likely to be
$ reproman run --follow -r local --sub slurm-launcher --orc datalad-no-remote --jp nodes=10 --jp jobs=10 -bp sub=$(all_bids_subjects) --bp space=MNI,native CMD '{p[sub]} {p[preproc]}'
where we would
--jp jobs=10
.was this fixed by #577?
rright! thank you @jdkent, lets close
TACC has launcher, does not support job arrays https://github.com/TACC/launcher
See https://frontera-portal.tacc.utexas.edu/user-guide/running/