fiber crosstalk overcorrects for brightest (backup) tiles

[sbailey]

iron/tiles/cumulative/82405/20211118/coadd-8-82405-thru20211118.fits has a bunch of spectra with a negative turnover at large wavelengths:

Figure out why and fix.

[abrodze]

It appears all exposures (all petals) from 2021118 have a turnover at long wavelengths, but it is most noticeable in sky fibers where the flux goes negative. I have not found this issue to be present in data from the preceding or following nights.

edit: the file here is from iron, but can confirm the issue persists with Jura

[araichoor]

if useful, looking here: https://data.desi.lbl.gov/desi/users/raichoor/spectro/iron/peramp-medians/peramp-median-iron-main-y1.pdf it seems it started on 20211114 until 20211121 (included), or maybe even 20211125 (included). I m looking at the purple points; those are below zeros for those dates; that s eye-balling. note that those are ~full moon nights (see plot titles).

(plot description here: https://github.com/desihub/desispec/issues/2193#issuecomment-1993045663)

[abrodze]

The sky fiber flux is mostly negative at all wavelengths in z & r and exhibits the negative slope at ~9200A for all cframes of tiles 82405 (exposures 00109351) and 82406 (00109355, 00109360, 00109363, 00109364, 00109366, 00109367, 00109370). All other tiles - all BRIGHT - from 20211118 seem ok.

Tracking these BACKUP tiles to the other nights they were observed shows this "negative trend" to be coupled with the tiles 84201-84209. Nights where at least one of these tiles was observed include 20211030, 20211109, 2021115, 20211118, 20211119, and 20211120.

[sbailey]

Debugging with @abrodze, we traced this to the fiber cross-talk subtractive correction introduced in PR #1138. Under normal conditions, this correction helps protect faint targets from being polluted by neighboring bright targets (see PR #1138 for examples). Under the extremely bright conditions of backup tiles where every fiber is very bright, the correction can become too large, leading to negative flux, especially at wavelengths>9000A where the correction is largest. This can be see by processing special backup tile 82405 night 20211118 expid 109363 with and without cross-talk correction:

cd /global/cfs/cdirs/desi/spectro/redux/jura/exposures/20211118/00109340
desi_process_exposure \
  -i frame-z8-00109340.fits.gz \
  --fiberflat fiberflatexp-z8-00109340.fits.gz \
  --sky sky-z8-00109340.fits.gz \
  --calib fluxcalib-z8-00109340.fits.gz \
  -o $SCRATCH/cframe-z8-00109340-default.fits

desi_process_exposure \
  -i frame-z8-00109340.fits.gz \
  --fiberflat fiberflatexp-z8-00109340.fits.gz \
  --sky sky-z8-00109340.fits.gz \
  --calib fluxcalib-z8-00109340.fits.gz \
  -o $SCRATCH/cframe-z8-00109340-noxtalk.fits --no-xtalk

Plotting the flux of the sky-subtracted xtalk-corrected flux-calibrated sky fibers:

There is clearly room for improvement in this correction on the brightest tiles, but that comes at the risk of making the correction worse for the main survey tiles which will require careful checking, so we will not change this in a rush just before Kibo.

[sbailey]

Plotting code for future reference:

import os
from desispec.io import read_frame
from scipy.ndimage import median_filter

cf1 = read_frame(os.path.expandvars('$SCRATCH/cframe-z8-00109363-default.fits'))
cf2 = read_frame(os.path.expandvars('$SCRATCH/cframe-z8-00109363-noxtalk.fits'))

isky = np.where(cf1.fibermap['OBJTYPE'] == 'SKY')[0]

plt.clf()
plt.subplot(211)
for i in isky:
    plt.plot(cf1.wave, median_filter(cf1.flux[i], 15), 'k-', alpha=0.05)
plt.ylim(-5,5)
plt.axhline(0)
plt.title('z8 sky fiber residuals\nspecial backup tile 82405 night 20211118 expid 109363')
# plt.title('z8 sky fiber residuals\main bright tile 23993 night 20211118 expid 109340')
plt.ylabel('with xtalk correction')

plt.subplot(212)
for i in isky:
    plt.plot(cf2.wave, median_filter(cf2.flux[i], 15), 'k-', alpha=0.05)
plt.ylim(-5,5)
plt.axhline(0)
plt.ylabel('no xtalk correction')
plt.xlabel('Wavelength')

plt.show()