search

igrins / plp

IGRINS pipeline package v3
18 stars 16 forks source link

Readout pattern removal slightly worse in qlook branch than v2.2.0 #22

Closed kfkaplan closed 5 months ago

kfkaplan commented 1 year ago

I have downloaded and tested the version of the IGRINS PLP that is in the qlook branch. For the most part it works great but I have noticed a few issues. One of the issues is that the readout pattern removal seems to be slightly worse in the qlook version when compared to the older version of the PLP (v2.2.0). I have attached a few animated gifs comparing output from the old and new and you can see that some of the columns are not as well removed in the qlook branch version, while in v2.2.0 the pattern removal looks really good. I tried to see what might be going on in destriper.py but I cannot find any differences in the algorithms that might be the cause of this. I wonder if it could have something to do with running in python 3 vs. python 2. pattern pattern_2

leejjoon commented 1 year ago

I believe I did some refactoring of the code for the readout pattern removal and I may have accidentally changed the default behavior. I will check.

ericasaw commented 7 months ago

@leejjoon just wanted to update you on how this is going! In addition to this background change, we noticed our most recent data from Gemini South has large noise artifacts (maybe from the detector vibrating while the image is read out?). Apparently these artifacts have always been in Gemini South raw frames but had little to no effect on data quality (in the old pipeline v2.2.0). In the new pipeline (qlook branch and derivatives) the background method implemented cannot remove these signals and the noise propagates into final science spectra.

Since the old pipeline's background removal is very robust to both removing background and the readout pattern from Gemini South we reimplemented it into the reimplement_cr_reject branch @kfkaplan has been working on. The background in all of the orders has improved significantly since reimplementing the old background removal code and we plan on retaining that method in the IGRINS pipeline for the plp V3 release.

kfkaplan commented 5 months ago

After an extensive investigation, we figured out the v2.2.0 PLP pattern removal worked quite well. We have reimplemented this original pattern removal code, but also run the the new pattern removal code as well. Now the pattern removal uses both algorithms. The end result is a slight improvement VS the v2.2.0 of the PLP.