Re-adjust wavelength solution on sky lines

[julienguy]

• Teststand data analysis shows drifts of traces of 0.2 - 0.5 pixel/degree Celsius.
• Can be predicted, but will not necessarily be sufficient (complex because of different time scales, also depends on the temperature probes location and reliability).
• We should develop an alternative based on the analysis of sky lines on science exposures.
• Ideally it should be done in 2D to also re-adjust the trace locations.
• However a 1D optional analysis, while the sky model is computed might turn out useful (would require 1D spectro-perfectionism to keep the original wavelength grid, which means resampling).

[sbailey]

+1 for doing this in 2D to re-adjust the trace locations for the PSF model for each science exposure prior to extractions. If it is only an xy offset that should be straightforward. If we also have to include degrees of freedom for scales and rotation, that could get problematic in the blue with fewer lines to constrain the fit.

Alas, our dreams of a perfectly stable spectrograph where we can use the exact same PSF for the entire night may be over.

[julienguy]

Work in progress in branch traceshift.

[julienguy]

The trace shifts are implemented in branch 'traceshift'.

usage: desi_compute_trace_shifts [-h] --image IMAGE --psf PSF [--lines LINES]
                                 [--spectrum SPECTRUM] --outpsf OUTPSF
                                 [--outoffsets OUTOFFSETS] [--degxx DEGXX]
                                 [--degxy DEGXY] [--degyx DEGYX]
                                 [--degyy DEGYY] [--nfibers NFIBERS]

Measure trace shifts from a preprocessed image and an input psf, and writes
the modified trace coordinates in an output psf file to be used for
extractions. Two methods are implemented. 1) cross-correlation : dx shifts are
measured from cross-dispersion profiles of traces. dy shifts (wavelength
calibration) are measured in two steps, an internal calibration determined
from the cross-correlation of fiber spectra obtained from a resampled boxcar
extraction with their median, and the final wavelength calibration is obtained
from the cross-correlation of the median fiber spectrum (after a second boxcar
extraction) with an external spectrum (given with --spectrum option). This
method is efficient for measuring trace shifts on science exposures with a
large sky background. 2) forward model : dy,dy shifts are determined
simultaneously by a forward modeling of the image around a given external list
of lines (--lines option). This method is in principle statistically optimal,
but it is slow and cannot be applied to blended and broad sky lines. It is
useful to shift traces from arc lamp images (though specex does the same thing
in C++).

optional arguments:
  -h, --help            show this help message and exit
  --image IMAGE         path of DESI preprocessed fits image (default: None)
  --psf PSF             path of DESI psf fits file (default: None)
  --lines LINES         path of lines ASCII file. Using this option changes
                        the fit method. (default: None)
  --spectrum SPECTRUM   path to a spectrum ASCII file (e.g. the DESIMODEL sky
                        spectrum) (default: None)
  --outpsf OUTPSF       path of output PSF with shifted traces (default: None)
  --outoffsets OUTOFFSETS
                        path of output ASCII file with measured offsets for QA
                        (default: None)
  --degxx DEGXX         polynomial degree for x shifts along x (default: 2)
  --degxy DEGXY         polynomial degree for x shifts along y (default: 2)
  --degyx DEGYX         polynomial degree for y shifts along x (default: 2)
  --degyy DEGYY         polynomial degree for y shifts along y (default: 2)
  --nfibers NFIBERS     limit the number of fibers for debugging (default:
                        None)

The method based on cross-correlations with an external sky spectrum was tested on the oak pixel level simulations.

Example at NERSC:

desi_compute_trace_shifts --image /project/projectdirs/desi/spectro/sim/oak/20160714/pix-r0-00000008.fits --psf  /project/projectdirs/desi/spectro/redux/oak1/calib2d/psf/20160714/psfnight-r0.fits  --outpsf shifted-psf-r0.fits --outoffsets offsets-r0-00000008-0.txt --spectrum desimodel/spec-sky.dat

After verifying the measured trace shifts where <0.05 pixels everywhere on the CCD, tests were performed with PSFs traces intentionally offset. The results are shown below.

• red curves: offsets from true traces with the incorrect input PSF
• blue curves: offsets from true traces after running desi_compute_trace_shifts.

The maximum residual offset is ~0.05 pixel, after a 2D polynomial fit of degree 2 as a function of x and y, for both dx and dy. The code ensures the statistical error is <0.05 pixel everywhere by iteratively reducing the polynomial degrees if needed.

Fits are done on all fibers, not just the sky. It certainly fails for standard stars, but there is sufficient redundancy to detect the outliers. We need however to check whether this still works for the bright time survey (the simulated exposure I used to develop this is a "dark" exposure).

b0

r0

z0

The second method, based on a forward model, was successfully tested on HgAr arc lines from teststand data. Tests on oak simulations show this second method is not well suited for the fit of sky lines.