desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
35 stars 24 forks source link

Pipeline call autocalib_fiberflat with --solve-gradient option for selected nights #2189

Closed sbailey closed 4 months ago

sbailey commented 5 months ago

Add hooks and override files to the pipeline to call desi_autocalib_fiberflat with the --solve-gradient option for a list of nights with known gradients (likely due to calib screen misalignment with the telescope). From @rongpu and @sybenzvi in #2172 these nights are:

20230925, 20231001, 20231002, 20231003, 20231004, 20231005, 20231006, 20231007, 20231008, 20231009, 20231010, 20231011, 20231012, 20231013, 20231014, 20231015, 20231203, 20231204, 20231205, 20231207, 20231214, 20231216, 20231217, 20231218, 20231219, 20231220, 20231221, 20231228, 20231229, 20231230, 20231231, 20240101, 20240102

Do this after PR #2187 is merged.

sbailey commented 5 months ago

Further details -- I think this would look like an override_NIGHT.yaml entry like

## 20230925 offset between telescope and dome screen so correct for fiberflat gradient
calibration:
    autocalib_fiberflat:
        solve_gradient: True 

which would propagate into desi_proc_joint_fit calling desispec.scripts.autocalib_fiberflat.main with an additional option --solve-gradient. Note that desispec.scripts.autocalib_fiberflat.main takes a list of strings as if it was constructing a command line for subprocess, and thus it needs a string '--solve-gradient'; it is not a function with key/value args that would need main(..., solve_gradient=True). We do not need to specify a reference night, nor does any reference night need to be processed before the night with the --solve-gradient override since the reference fiberflats are in $DESI_SPECTRO_CALIB itself.

Currently the override files are handled in PR #2187 desi_proc_night at the pipelining level. I think this is the first case where we need to be passing an option down into the algorithmic level.

akremin commented 4 months ago

As noted in Issue #2174, these 33 nights are now covered by override files specifying that we should set --solve-gradient in autocalib_fiberflat.

A    202401/override_20240101.yaml
A    202401/override_20240102.yaml
A    202309/override_20230925.yaml
A    202310/override_20231001.yaml
A    202310/override_20231011.yaml
A    202310/override_20231003.yaml
A    202310/override_20231013.yaml
A    202310/override_20231005.yaml
U    202310/override_20231015.yaml
A    202310/override_20231007.yaml
A    202310/override_20231009.yaml
A    202310/override_20231010.yaml
A    202310/override_20231002.yaml
A    202310/override_20231012.yaml
A    202310/override_20231004.yaml
A    202310/override_20231014.yaml
A    202310/override_20231006.yaml
A    202310/override_20231008.yaml
A    202312/override_20231203.yaml
A    202312/override_20231221.yaml
A    202312/override_20231231.yaml
A    202312/override_20231205.yaml
A    202312/override_20231207.yaml
A    202312/override_20231217.yaml
A    202312/override_20231219.yaml
A    202312/override_20231229.yaml
A    202312/override_20231220.yaml
A    202312/override_20231230.yaml
A    202312/override_20231204.yaml
A    202312/override_20231214.yaml
U    202312/override_20231216.yaml
A    202312/override_20231218.yaml
A    202312/override_20231228.yaml