legacysurvey / legacypipe

Image reduction pipeline for the DESI Legacy Imaging Surveys, using the Tractor framework
BSD 3-Clause "New" or "Revised" License
37 stars 22 forks source link

Moving dust/condensation in Bok images #367

Open djschlegel opened 5 years ago

djschlegel commented 5 years ago

The dr8c reductions show a strange set of pixels surrounding a masked region. Identified by Arjun in the viewer here: http://legacysurvey.org/viewer-dev?ra=138.2993&dec=33.3159&zoom=12&layer=dr8c-90p-mos&bricks

This appears to originate from a masked region in the upper right of HDU#1 of 90prime/BOK_CP/CP20160111/ksb_160112_102913_ooi_g_v1.fits.fz

Some discussion between Arjun & Frank on decam-chatter 4/24/2019 that this was due to transient dust on the filter for the night of 2016-01-11. Example image from the night below.

bok

dstndstn commented 5 years ago
1

so it looks like this region is mostly masked, but the mask isn't quite big enough.

fvaldes commented 5 years ago

The problem starts in middle of night 16-01-11 to the end of the night. Not present on the following night 16-01-12

Starts in middle of night on 16-01-11

ksb_160112_051135_ooi_g_v1: first faint appearance ksb_160112_055147_ooi_g_v1; becomes fairly noticible ksb_160112_063005_ooi_g_v1: masking starts ksb_160112_071932_ooi_g_v1: second spot starts to be visible ksb_160112_103203_ooi_g_v1: additional spots, second spot causing halo ksb_160112_122723_ooi_g_v1: last BASS exposure 6 spots

Not present on 16-01-12

zouhu commented 5 years ago

The weather that night is quite good. The weather record was interrupted for a few hours, but generally, the wind speed was relative high and the relative humidity steadily decreased from 70% to 20%. We didn't find any record in our log file. You can find the weather plots here

You might feel funny: there were moths burst into the camera, especially when in Summer. You can even see a series of images and find they were moving. Several images in 2017 were "polluted". See attached image (black-white moth photo in the bottom right corner :D ).

Screen Shot 2019-05-06 at 4 39 38 PM
fvaldes commented 5 years ago

Dear Hu and all,

Thanks for the information. The moth example does look like the major feature of the second half of the night. I am not an expert on what to expect so I don’t understand why the feature gets stronger with time and other features appear and also get stronger with time. I also don’t know why the shadows consist of rings since I’ve seen moth shadows on other cameras that were more like an insect body. However, I suppose it depends on the surface distances.

I produced an animation of the night which I think you all might find interesting. It also gives an appreciation for the BASS data as produced by the NOAO pipeline and the distortion feature on the lower right CCD. While the individual frames were not specifically produced at the same greyscale stretch bit on average they are pretty similar and you can believe the strength of the out of focus features does increase with time.

Time lapse animation of BASS 2019-01-11: https://www.noao.edu/noao/staff/fvaldes/BASS20160111/20160111anim.gif https://www.noao.edu/noao/staff/fvaldes/BASS20160111/20160111anim.gif

You can download or play on your browser.

Frank

On May 6, 2019, at 1:42 AM, Hu Zou notifications@github.com wrote:

The weather that night is quite good. The weather record was interrupted for a few hours, but generally, the wind speed was relative high and the relative humidity steadily decreased from 70% to 20%. We didn't find any record in our log file. You can find the weather plots here

http://batc.bao.ac.cn/BASS/data/media/observation/observation/imagestats/7399/weather.png http://batc.bao.ac.cn/BASS/data/media/observation/observation/imagestats/7399/weather.png http://batc.bao.ac.cn/BASS/data/media/observation/observation/imagestats/7399/image_quality.png http://batc.bao.ac.cn/BASS/data/media/observation/observation/imagestats/7399/image_quality.png You might feel funny: there were moths burst into the camera, especially when in Summer. You can even see a series of images and find they were moving. Several images in 2017 were "polluted". See attached image (black-white moth photo in the bottom right corner :D ).

https://user-images.githubusercontent.com/10877616/57214873-dd428f00-701d-11e9-9278-c7738472104c.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/legacysurvey/legacypipe/issues/367#issuecomment-489548931, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIBNKAK4UX3W553LYM7AJLPT7VOHANCNFSM4HIYA6RA.

zouhu commented 5 years ago

Dear Frank and all, It is more likely that the moth got closer to the surface as the telescope moved. I just recalled the observing, I was me observing at the Bok that time. And it was the first time we found this problem. Indeed it was a moth flying into the filter wheel. After that time, the BASS observers also encountered several such insect invasions. Some one even saw the moth moving as shown in the frames. Usually, we need to rotate the filter wheel several times in order to remove the insect. But any ways, these events only occurred occasionally ( several times, more frequently in summer).

Maybe the DESI needs to consider to avoid such insects entering into some instruments.

moustakas commented 4 years ago

@fvaldes @schlafly @djschlegel can one of you take a look and see if this is still an issue with the reprocessed images?

fvaldes commented 4 years ago

Forwarding to D. Herrera as well. I think the question, as a matter of a resolution, is whether the moths are masked (bandit moths). This may not always be the case and the only workable solution may be to exclude the CCDs where this occurred and were not automatically masked. Still a manual job unfortunately.

Frank

On Dec 23, 2019, at 10:48 AM, Moustakas notifications@github.com wrote:

@fvaldes https://github.com/fvaldes @schlafly https://github.com/schlafly @djschlegel https://github.com/djschlegel can one of you take a look and see if this is still an issue with the reprocessed images?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/legacysurvey/legacypipe/issues/367?email_source=notifications&email_token=AAIBNKDE36YVDDH4DUJEC33Q2D2WZA5CNFSM4HIYA6RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHRTJ6A#issuecomment-568538360, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIBNKHMWRVOW62JLFA6NNLQ2D2WZANCNFSM4HIYA6RA.

fvaldes commented 4 years ago

Hi,

I don’t see any info on what images/dates are you referring to. There were a couple of cases of eventual moving moths and in the previous CP versions, they seem properly masked. At any rate, I didn’t not get the list of images in the forwarding. I assume this is different from the under-masked trails issue.

David

On Dec 24, 2019, at 11:07, Frank Valdes valdes@noao.edu wrote:

Forwarding to D. Herrera as well. I think the question, as a matter of a resolution, is whether the moths are masked (bandit moths). This may not always be the case and the only workable solution may be to exclude the CCDs where this occurred and were not automatically masked. Still a manual job unfortunately.

Frank

On Dec 23, 2019, at 10:48 AM, Moustakas <notifications@github.com mailto:notifications@github.com> wrote:

@fvaldes https://github.com/fvaldes @schlafly https://github.com/schlafly @djschlegel https://github.com/djschlegel can one of you take a look and see if this is still an issue with the reprocessed images?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/legacysurvey/legacypipe/issues/367?email_source=notifications&email_token=AAIBNKDE36YVDDH4DUJEC33Q2D2WZA5CNFSM4HIYA6RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHRTJ6A#issuecomment-568538360, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIBNKHMWRVOW62JLFA6NNLQ2D2WZANCNFSM4HIYA6RA.

moustakas commented 4 years ago

@djschlegel can this ticket be closed? Otherwise please push to DR10.