It appears that nightqa raises an error if rerun without --recompute and an output exists. This may be understandable for the HTML file, but for all others it should just skip over them and continue. That is the operating mode for all other software in desispec and allows checkpointing when running at scale.
INFO:night_qa.py:176:get_dark_night_expid: proctable_fn = /global/cfs/cdirs/desi/spectro/redux/loa/processing_tables/processing_table_loa-20240319.csv
WARNING:night_qa.py:178:get_dark_night_expid: no /global/cfs/cdirs/desi/spectro/redux/loa/processing_tables/processing_table_loa-20240319.csv found; returning None
INFO:night_qa.py:234:get_morning_dark_night_expid: exptable_fn = /global/cfs/cdirs/desi/spectro/redux/loa/exposure_tables/202403/exposure_table_20240319.csv
WARNING:night_qa.py:243:get_morning_dark_night_expid: found zero exposures with OBSTYPE=dark and COMMENTS='|' and abs(EXPTIME-1200)<1 in expable_fn; returning None
INFO:desi_night_qa:99:main: ('prod', '/global/cfs/cdirs/desi/spectro/redux/loa')
INFO:desi_night_qa:99:main: ('night', 20240319)
INFO:desi_night_qa:99:main: ('group', 'cumulative')
INFO:desi_night_qa:99:main: ('outdir', '/global/cfs/cdirs/desi/spectro/redux/loa/nightqa/20240319')
INFO:desi_night_qa:99:main: ('css', PosixPath('/global/common/software/desi/perlmutter/desiconda/20240425-2.2.0/code/desispec/0.67.0/lib/python3.10/site-packages/desispec/data/qa/night\
qa.css'))
INFO:desi_night_qa:99:main: ('recompute', False)
INFO:desi_night_qa:99:main: ('compute_missing_only', False)
INFO:desi_night_qa:99:main: ('steps', 'html,dark,morningdark,badcol,ctedet,ctedetrowbyrow,sframesky,tileqa,skyzfiber,petalnz')
INFO:desi_night_qa:99:main: ('nproc', 64)
INFO:desi_night_qa:99:main: ('dark_bkgsub_science_cameras', 'b')
INFO:desi_night_qa:126:main: steps to be done: html,dark,morningdark,badcol,ctedet,ctedetrowbyrow,sframesky,tileqa,skyzfiber,petalnz
INFO:desi_night_qa:130:main: will create /global/cfs/cdirs/desi/spectro/redux/loa/nightqa/20240319/nightqa-20240319.html
INFO:desi_night_qa:130:main: will create /global/cfs/cdirs/desi/spectro/redux/loa/nightqa/20240319/dark-20240319.pdf
ERROR:desi_night_qa:136:main: /global/cfs/cdirs/desi/spectro/redux/loa/nightqa/20240319/dark-20240319.pdf already exists, and args.recompute = False
Traceback (most recent call last):
File "/global/common/software/desi/perlmutter/desiconda/20240425-2.2.0/code/desispec/0.67.0/bin/desi_night_qa", line 210, in <module>
main()
File "/global/common/software/desi/perlmutter/desiconda/20240425-2.2.0/code/desispec/0.67.0/bin/desi_night_qa", line 137, in main
raise ValueError(msg)
ValueError: /global/cfs/cdirs/desi/spectro/redux/loa/nightqa/20240319/dark-20240319.pdf already exists, and args.recompute = False
It appears that nightqa raises an error if rerun without
--recomputeand an output exists. This may be understandable for the HTML file, but for all others it should just skip over them and continue. That is the operating mode for all other software in desispec and allows checkpointing when running at scale.