Opening the PR with a branch from @akremin to add --dry-run-level N functionality. I added additional start/end log messages and help string info in commit c4b03ba7e012d0da5cd6c1a7aea1cf943ef7328a .
e.g. for Jura,
desi_healpix_redshifts --survey main --program bright --bundle-healpix 10 --dry-run-level 3
...
INFO:healpix_redshifts.py:166:main: Dry run: would have submitted 1791 batch scripts
INFO:healpix_redshifts.py:169:main: All done at Tue May 28 14:47:04 2024; duration 5.25 minutes
This allows planning the number of jobs that will be added to the queue prior to launching them so that we don't hit the 5k submit limit. I confirmed that running in --dry-run-level 3 will not attempt to generate any files (I ran it under my own account, so it would have crashed if it had tried).
Opening the PR with a branch from @akremin to add
--dry-run-level Nfunctionality. I added additional start/end log messages and help string info in commit c4b03ba7e012d0da5cd6c1a7aea1cf943ef7328a .e.g. for Jura,
This allows planning the number of jobs that will be added to the queue prior to launching them so that we don't hit the 5k submit limit. I confirmed that running in --dry-run-level 3 will not attempt to generate any files (I ran it under my own account, so it would have crashed if it had tried).