desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
33 stars 24 forks source link

Add script to create override file #2214

Closed akremin closed 3 months ago

akremin commented 3 months ago

This adds a script called desi_create_override_file that will prompt the user for relevant information that can be included in an override file. Some of the information can be provided via the command line for easier scripting. This was used to create the override files needed to set the autocalib_fiberflat --solve_gradient flag for about a month's worth of nights in 2023 and 2024 (issue #2174). The files are in the proper format, which I tested by running desi_proc_night on a test night, which correctly propagated the --autocal-ff-solve-grad to desi_proc_joint_fit.

Note, because of the way the pyyaml module dumps information, the lists will be printed in long form rather than abbreviated bracket ([]) notation if linkcal is specified. This minor inconvenience was outweighed by the convenience of being able to script the generation of these files.

Important note: This is not intended to edit existing override files. If an existing override file is detected, it is moved out of the way by inserting a timestamp. The contents of the old override file are NOT used in the generation of the new override. A warning is printed about this, but it is important for the user to either include all relevant override information again to generate the new override via the script (preferred), or copy over the relevant entries from the old file manually after generating the new entries.

sbailey commented 3 months ago

For --linkcal and --ff-solve-grad let's use

parser.add_argument('--linkcal', action=argparse.BooleanOptionalAction, help=...)
parser.add_argument('--ff-solve-grad', action=argparse.BooleanOptionalAction, help=...)

which will result in the args being None if not specified, but allow the user to specify --linkcal / --no-linkcal to specifically override in either direction and avoid the interactive prompts.

Suggested rewording: "NOTE: Remember to set the reference night in desi_spectro_calib" -> "NOTE: check that reference fiberflats in desi_spectro_calib are valid for gradient correction on YEARMMDD"

Other minor items:

akremin commented 3 months ago

Thank you for the useful suggestions. I have implemented all of them in the most recent commit.