desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
36 stars 24 forks source link

tile 80708 QSO failures: fiducials on? #1168

Open sbailey opened 3 years ago

sbailey commented 3 years ago

As reported by @Cyeche:

I was analyzing the QSOs of tile #60708, processed with cascades /global/cfs/cdirs/desi/spectro/redux/cascades/tiles/80708/20210214/ I found too many QSOs with a redshift z~2.77. I don’t know the origin of the problem, but this is due to a big bump on the spectra which is identified as a broad Ly-alpha line by RR

image image image

This looks like a flat fielding error of the collimator absorption feature, but a first glance at the calibs that night doesn't reveal anything obvious.

julienguy commented 3 years ago

I suspect it's contamination by another light source. See https://nightwatch.desi.lbl.gov/20210214/00075868/preproc-b4-00075868-4x.html . One possibility is that some fiducials were still on.

sbailey commented 3 years ago

For comparison, sky spectra from 20191030/22995 when we know the fiducials were still on: image

julienguy commented 3 years ago

Very likely a fiducial was ON. It affected exposures 75868,75869,75870,75873,75874 the last ones been with twilight. Example FP image of flux around 4700A. Figure_1 The only thing we can do is declare bad those exposures.

araichoor commented 3 years ago

in case it s useful (see [desi-data 5341]): I ve done my z vs. fiber plots on a per-night basis. looks like at least the following nights have a ~similar feature: 20201222 : https://data.desi.lbl.gov/desi/users/raichoor/cascades-z-fiber/cascades-z-fiber-dchi2min9-20201222.png 20201223 : https://data.desi.lbl.gov/desi/users/raichoor/cascades-z-fiber/cascades-z-fiber-dchi2min9-20201223.png 20210111 : https://data.desi.lbl.gov/desi/users/raichoor/cascades-z-fiber/cascades-z-fiber-dchi2min9-20210111.png 20210214: https://data.desi.lbl.gov/desi/users/raichoor/cascades-z-fiber/cascades-z-fiber-dchi2min9-20210214.png

note that I don t test all nights. I also highlight in that email other features...

julienguy commented 3 years ago

Based on Anand's preselection, I am detecting fiducials ON in exposures. This is not an automated search. So it's probably not complete.

20201222    69435,69436,69437,69438,69440,69441,69442,69444,69445,69446,69447,69449,69450,69451,69452,69453,69455,69456,69458,69459,69460
20201223    69628,69629,69630,69631,69632
20210111    71870
20210214    75823,75824,75825,75826,75827,75828,75829,75830,75831,75832,75835,75836 75868,75869,75870,75873,75874
araichoor commented 3 years ago

for info, I ve been looking a bit at this: looks like there also is an issue with light from bright stars.

some of the cases listed by Julien are because of that. for instance: cascades-20210214-00075827-cframe-4600w4800 on the left-side plot, I color-code the cframe mean flux in 4600A-4800A, divided by the median value over all the fibers.

another striking case: cascades-20210215-00076029-cframe-4600w4800

I m generating similar plots for all sv1 science exposures; those are appearing here /global/cscratch1/sd/raichoor/desi-fiducial-check/ (should be finish in few mins).

sbailey commented 3 years ago

@araichoor interesting; indeed it appears that we need to refine our "fiducials-on" selection for the next run. (The current selection was done in a heroic last minute rush; I'm not complaining). Dealing with stellar scattered light is a different and trickier issue.

That's a nice view of the focal plane next to the imaging viewer which makes the correlation with bright stars obvious. When your plots are done, please move them to somewhere under /global/cfs/cdirs/desi/users/raichoor so that we can browse them from https://data.desi.lbl.gov .

In particular, I'm interested in looking at tile 80687 night 20210208 expid 75103 which was not flagged as bad even though it should also be polluted by bright star reflections.

Tile 80690 (your second example) was not excluded by the fidicuals-on cut for cascades.

Also mentioning @deisenstein and @paulmartini who have studied the impact of bright star reflections.

araichoor commented 3 years ago

I ve copied the files in $DESI_ROOT/users/raichoor/desi-fiducial-check. e.g.: https://data.desi.lbl.gov/desi/users/raichoor/desi-fiducial-check/cascades-20210208-00075103-cframe-4600w4800.png https://data.desi.lbl.gov/desi/users/raichoor/desi-fiducial-check/cascades-20210214-00075827-cframe-4600w4800.png you re correct, 75103 has nothing special. one difference maybe was that 75827 was observed in gray time (OBSCONDITIONS=2), whereas 75103 in dark time (OBSCONDITIONS=1)? here is the g-r vs. rmag sky values for all sv exposures, highlighting those two: image

deisenstein commented 3 years ago

For the GFA-reflection-ghost study, it was very effective to look at differences of narrow bands. So for the fiducials problem, where there is a known and narrow wavelength, I would suggest measuring that band and subtracting off a pair of nearby bands (avoiding the 4400A dichroic problem!). By balancing a redder & bluer control band, one can largely null out the flux from stars and their far PSF.

But the far PSF problems also need to get worked. Obviously we know the location of these stars, and so it seems that one ought to be able to fit some kind of residual here across the patches of affected fibers.

Identifying the location and magnitude of the brightest ~5 stars per tile might help us to prioritze this...

Thanks, Daniel

On Fri, Mar 5, 2021 at 1:25 PM araichoor notifications@github.com wrote:

I ve copied the files in $DESI_ROOT/users/raichoor/desi-fiducial-check. e.g.:

https://data.desi.lbl.gov/desi/users/raichoor/desi-fiducial-check/cascades-20210208-00075103-cframe-4600w4800.png

https://data.desi.lbl.gov/desi/users/raichoor/desi-fiducial-check/cascades-20210214-00075827-cframe-4600w4800.png you re correct, 75103 has nothing special. one difference maybe was that 75827 was observed in gray time (OBSCONDITIONS=2), whereas 75103 in dark time (OBSCONDITIONS=1)? here is the g-r vs. rmag sky values for all sv exposures, highlighting those two: [image: image] https://user-images.githubusercontent.com/61986357/110157191-c5183400-7d9c-11eb-9c71-97668d9cb1c9.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/desihub/desispec/issues/1168#issuecomment-791599320, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPQRGWK46DC7DEC3QYQTTTTCEOYHANCNFSM4YTWICUA .

sbailey commented 2 years ago

FTR: The specific exposures identified in this ticket have been flagged as bad prior to Fuji