Measure noise correlation in preprocessing

desihub / desispec

DESI spectral pipeline

BSD 3-Clause "New" or "Revised" License

33 stars 24 forks source link

Measure noise correlation in preprocessing #2097

Open julienguy opened 10 months ago

julienguy commented 10 months ago

Measure noise correlation along X (cross-dispersion, within a row) in overscan regions during preprocessing
Save one number per amplifier (like correlation over 4 pixel delta X separation)
Use this to adjust the inverse variance in the spectra because correlated noise over 3-4 pixels results in higher noise in the spectra than uncorrelated noise, (value TBD).

sbailey commented 10 months ago

This is most directly relevant for b1 data starting 20230805 (new CCD with correlated noise on amp A), but should be studied on earlier data and other CCDs too.

@julienguy please correct any details in the above statement.

I'm adding this to the Jura project since we may need this for the Y3 processing.

julienguy commented 10 months ago

We looked at this with @p-slash (Naim) and found that pixel correlations in a few amplifiers are very likely the cause of the variation of the 'eta' parameter with wavelength reported in https://ui.adsabs.harvard.edu/abs/2023arXiv230606316K/abstract figure 13.