search

legacysurvey / legacypipe

Image reduction pipeline for the DESI Legacy Imaging Surveys, using the Tractor framework
BSD 3-Clause "New" or "Revised" License
37 stars 22 forks source link

Cosmic ray masking on many BASS exposures masks many stars #334

Closed djschlegel closed 4 years ago

djschlegel commented 5 years ago

Many Bok exposures mark many stars as cosmic rays when the seeing is good. This is slightly tragic, as it wrecks some of our best Bok data.

We should either reprocess these exposures such that these stars are not marked as comics, or perhaps interpret the mask files to recover those pixels.

The attached file from Eddie has the fraction of stars flagged for each Bok exposure (converted to a text file).

Below is a list of some of the worst cases. FILENAME NSTAR NFLAGGED ksb_170422_082527_ooi_r_v1-star-photom.fits 3070 1134 ksb_180310_053809_ooi_g_v1-star-photom.fits 6785 2415 ksb_160216_044352_ooi_r_v1-star-photom.fits 4167 1588 ksb_160216_044013_ooi_r_v1-star-photom.fits 4104 1525 ksb_170501_053733_ooi_r_v1-star-photom.fits 3102 1424 ksb_170501_045722_ooi_r_v1-star-photom.fits 2889 1064 ksb_170422_070543_ooi_r_v1-star-photom.fits 2914 1144 ksb_170320_111730_ooi_r_v1-star-photom.fits 4165 1476 ksb_160711_071432_ooi_r_v1-star-photom.fits 9907 3677 ksb_170422_071946_ooi_r_v1-star-photom.fits 2752 1212 ksb_160711_070206_ooi_r_v1-star-photom.fits 9086 3293 ksb_170320_111257_ooi_r_v1-star-photom.fits 3930 1385 ksb_180118_073138_ooi_r_v1-star-photom.fits 3647 1760 ksb_170320_111045_ooi_r_v1-star-photom.fits 3735 1340

bokflaggedfraction.txt

djschlegel commented 5 years ago

This is a reminder that Frank did a reprocessing of one night, 2017-04-21, and we therefore have two versions of each of these files in BOK_CP/CP20170421 and BOK_TEST/CP20170421 . The DR8 file list has chosen the first reduction (v1 in BOK_CP) for 2 files on that night, and the reprocessing (v2 in BOK_TEST) for 225 files, even though both versions exist in all cases.

Here was the note that Frank sent on Nov 9, 2017, describing these test reductions: I have made changes to the Bok and Mosaic CPs to address the problem of misidentified stars as cosmic rays. The changes are: -- the cosmic ray algorithms are not run if the estimated FWHM is less than 3.5 pixels -- cosmic ray replacement by interpolation is not done for Bok data though the CR flag value of 5 is in the data quality file Because of the structure of the workflow the CR algorithms occur per CCD before WCS solutions and the seeing estimate, SEEINGP1, is normally good but occasionally can be a bit off. In the previous version the SEEINGP keyword was simply the average of the independent CCD estimates. Another change, which is independent of the CR changes above, is now to use only the sources identified as matching the photometric reference catalog (the Legacy PS1 stellar catalog) after the WCS solution over all CCDs. This should be more robust against outliers. The point is that the only star identification in the initial per CCD processing is just by assuming there is some peak at small values in measures of FWHM for all detected sources. Also for reasons of efficiency the FWHM measure is not any kind of non-linear model fit but a direct estimate of FWHM.

For the above night with 1.6 arcsec seeing, the fraction of masked pixels from the CP did not change by very much between the two sets of reductions. The mean fraction of masked pixels went from 4.0% to 3.5%. We should probably find some better test cases where more pixels were masked in the first place.

moustakas commented 5 years ago

344 completely turns off the CP cosmic-ray masking. We should run these images through the master branch (and maybe through zeropoints again) to see and study the impact.

moustakas commented 4 years ago

Fixed in reprocessing, thanks @fvaldes!