Check negative outliers in DR8b

legacysurvey / legacypipe

Image reduction pipeline for the DESI Legacy Imaging Surveys, using the Tractor framework

BSD 3-Clause "New" or "Revised" License

37 stars 22 forks source link

Check negative outliers in DR8b #267

Closed moustakas closed 4 years ago

moustakas commented 5 years ago

Something may be going amiss in the outlier masking, based on this piece of sky-- http://legacysurvey.org/viewer-dev?ra=137.4645&dec=33.1566&zoom=14&layer=dr8b-decam

dstndstn commented 5 years ago

Super weird thing is, I re-ran brick 1374p332 and the CRs were all masked correctly.

I'm wondering if it is related to resuming, whether from the outlier-mask files or from pickles.

In the dr8b logs, it looks like this brick ran several times:

first time, nothing printed
second time, stages tims, mask_junk and srcs finished; fitblobs started
third time, resumes from srcs, runs fitblobs
... a few more times, same, plugging away at fitblobs
finally, fitblobs finishes, plus coadds, wise_forced, and writecat.

So then it doesn't look like it read from outlier-mask files, and the tims-mask_junk-srcs section finished in one shot. I don't get it.

And dr8b pickles for this brick are now blown away....

djschlegel commented 5 years ago

I can confirm that these cosmics are indeed properly identified in the outlier-mask files, but then those outlier masks don't appear to be used.

One nice long event to look at is in HDU#4 (S15) of image /global/homes/d/djschleg/cosmo/staging/decam/DECam_CP/CP20170228/c4d_170301_025929_ooi_r_v1.fits.fz

Trim this image to [14:1283,1903:4079] to match the subregion in dr8b/decam/metrics/137/1374p332/outlier-mask-decam-624490-S25.fits.fz

dstndstn commented 5 years ago

Fixed in 56c6d9f88f965a14e10516e09c4b7f5037bc71e5 (dammit)

dstndstn commented 5 years ago

PS, this affects reading existing outlier mask files from disk. I therefore expect that what happened was the first run, despite producing no logging, actually did create these mask files, and then the second run read from them.

It could be worth finding & re-running the affected bricks...

moustakas commented 5 years ago

A related issue is to check how often the "negative outlier" bit gets set. We want to be sure that a particularly bright artifact on a single image doesn't change the mean enough that the other images in the stack get flag as outlier negative pixels.

We should be able to check this in dr8b.

moustakas commented 4 years ago

I'm declaring victory on this issue.