Drift Correction - First Round

[awhoward]

We will sprint on this.

Components needed:

• Pipeline primitive to return a wavelength solution at a specific time. This should be in a new class in modules/wavelength_cal/src/wavelength_cal.py called WaveInterpolate. There will also be a method in a new class called WaveInterpolation in modules/wavelength_cal/src/alg.py. @bjfultn will develop this. He should start a new branch (feature/drift_corr) ASAP and merge it to develop. @RussLaher and @shalverson can then use these classes.
• Query of masters database for bracketing wavelength solutions. Returns two paths. This should be a method of WaveInterpolation so that it can be run in other environments. The algorithm should look backward first and return the best WLS (order: LFC, Thar), then look forward in time and return a matching type of WLS, if available, and return nothing if unavailable. @RussLaher will develop this.
• The header of L1 files should be modified to include keywords to note the bracketing WLS files and directories (currently, WLSFILE is truncated; it should be split). Let's also add a keyword called WLSNOTE that can have comments about the WLS interpolation used (e.g., "linear interpolation").
• Method to read two WLSs (from query) and linearly interpolate between them. This should be a method of WaveInterpolation. To start let's just linearly interpolate between matching L1 pixels in the two WLSs. Note this bracketing wavelength solutions are only available when reprocessing data. We will later need to develop an extrapolation method (after understanding the characteristics of the drift correction), and a method for reprocessing (perhaps after one day?). @shalverson will work on this.
• Make sure that the time associated with each master WLS L1 file is the midpoint of the stack. Consider if other keywords in the master WLS L1 file are inaccurate because they are copied from another L1 file. @RussLaher will work on this.

Testing (@howardisaacson):

• Identify a set of ~3-5 standard stars (and SoCal observations?). Perhaps only June 2023 - January 2024 to avoid times when KPF experienced thermal perturbations. @shalverson will do some initial work to identify files.
• Set sandbox with a /testdata directory to store outputs.
• Make a copy of the standard recipe, configure it to use /testdata, and run it.
• Analyze the RV time series of the standard stars in /testdata. This could be accomplished with AnalyzeTimeSeries() or using other methods.

[shalverson]

As a summary to reference later, here is my current sandbox approach for drift-correcting the stellar spectra (copied from slack thread, but listed here for convenience):

For the stellar RV calculation: -Select only observations that have both an evening LFC_WLS master and a morning LFC_WLS master with the same UT date. -Compute the stellar RV using the evening master LFC WLS, only including orders that are LFC-calibrated (>order index 11 on the Green).

For the LFC drift tracking: -Compute every day’s LFC master ‘RV’ (evening and morning) using the evening LFC_WLS master solution only. For evening masters, the RV should be very close to 0 since the WLS is derived from this file. -Linearly interpolate each day’s evening-to-morning LFC RV drift onto the timestamps of the stellar observations.

• Subtract the interpolated LFC RV from the stellar RV.

Stellar spectra quality cuts: -Outlier rejection at >1000 m/s off of the median target RV. This does nothing for some targets, for others it removes some clear outliers that should be followed up on jump. -Rejection of points where the RV error is >2x the median RV error in the timeseries (i.e. if the flux level is a factor of 4x lower than the median observation for that target, reject that point.). This also typically affects very few points.