desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
33 stars 24 forks source link

select calib dark with most cameras; fixes 20231028 for Jura #2253

Closed sbailey closed 1 month ago

sbailey commented 1 month ago

This PR fixes #2247 where the pipeline was generating a ccdcalib job for camword a01456789 but later needing CTE corrections for z3 not included in that camword.

The underlying reason was that the job camword is tied to the selected dark, and select_calibrations_to_proc was picking the first calib dark instead of the one with the most available cameras.

With this PR, desi_proc_night -n 20231028 --dry-run-level 1 now selects a different dark and generates a ccdcalib job for all cameras.

20231028 is the only night in Jura that I'm aware of that is impacted by this issue, though it is possible that other nights would pick a different dark with this branch.

Additional notes:

sbailey commented 1 month ago

Good idea. I updated select_calib_darks to return a Table instead of an index. Currently this is a Table of length 1, but in general in the future it could be a longer table to I made the function name plural and return a Table not a single Row object. This also simplifies its use with vstack with the other calib tables.

sbailey commented 1 month ago

Design note: as originally implemented, select_calib_darks would raise a ValueError if no good darks were found. That breaks the updated test_missing_cals test when merged into main, so I updated this to return a length-0 table instead. Yay for unit tests; the updated test in PR #2254 just prevented us from accidentally re-breaking that code with this PR.