desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
33 stars 24 forks source link

Add hooks for autocalib_fiberflat flag into override file and pipeline #2208

Closed akremin closed 3 months ago

akremin commented 3 months ago

This is the code changes to resolve issue #2189 by introducing new allowed override file keywords and implementing the code to interpret those and pass them to autocalib_fiberflat inside desi_proc_joint_fit. This is done by introducing a new flag to desi_proc_joint_fit that when set, runs autocalib_fiberflat with the --solve-gradient flag.

The new override file format differs from the recommendation in the issue. The format is:

calibration:
    nightlyflat:
        extra_cmd_args: ['--autocal-ff-solve-grad'] 

(Note for the future that originally "extra_cmd_args" was "flags" but after the comment below was updated. I have edited this so that the example is correct for future reference.)

This follows the logic from the calibration linking, where jobs that are only performed once per night can be uniquely identified by their JOBDESC (job description) in the override files. This approach breaks down for per-exposure or per-tile specifications but the simplicity here outweighed that for me. When we introduce per-exp or per-tile overrides, we will have to determine how to approach that and allow the override to have multiple meanings for second (and third and fourth) levels. That is left for a future day.

I have also added a unit test which ensures that the nightlyflat script being produced includes the --solve-gradient flag when --autocal-ff-solve-grad is specified as a flag in the override file.