A weird feature in raw images overlapped with a sky line for wavelength calibration

[hanaeinami]

I found a sudden increase of RMS near the center of one of the slits (slit 17) when I ran Wavelength.fit_lambda for my obsfiles. It appears that this is caused by a weird feature in the raw images (bad pixels?) that happens to lie on a strong sky emission line which is used for wavelength calibration.

I've attached a plot which shows the sudden jump of the RMS (vs. pixel and slit numbers) and a screenshot of a 2D image of the feature that caused this issue. The image is combined science images made with Wavelength.imcombine. This feature is extended around (x,y) = (266, 493). I think that because it happens to lie on a strong sky emission line, it messed up the wavelength fitting. I also found a similar feature at around (1320,1105), (1097, 575), (1008, 223).

These features appear in all of the data during our observing run in April 2014, and also in the data taken in September 2014. So I guess that these could have originated from the detector? So far, the wavelength calibration problem only happens when these features happen to overlap with a sky line, but if possible, it'd nice to be able to mask out this feature for the next version of the DRP.

Also, a minor bug report:

The slit numbers in the output of Wavelength.fit_lambda don't look right. For example, for the same mask above, Wavelength.fit_lambda says that:

S01 = pixels 7-77
S02 = pixels 1907-2031
S03 = pixels 1772-1894
S04 = pixels 1684-1762
     ...
S21 = pixels 221-298
S22 = pixels 132-210
S23 = pixels 88-122

On the other hand, Background.handle_background seems to handle the slit numbers correctly (has a -1 offset in the slit number, but that's OK):

slit 0 [2034,1904]
slit 1 [1896,1769]
slit 2 [1764,1681]
    slit 3 [1675,1593]
slit 4 [1589,1505]
 …
slit 21 [124,85]
slit 22 [79,4]

I've attached a screenshot below to visualize this. The lines and numbers at the top of the figure indicate the slit positions and slit numbers. The green ones are from the Wavelength.fit_lambda outputs and the orange are from Background.handle_background above.

Thank you, Hanae Inami

[nickkonidaris]

Hi Hanaei,

The bad pixel volcano that you circled in the image is like seeing an old friend again. Is it weird that I knew that feature was masked out?

Still, I have to admit that I checked. The volcano is in the bad pixel mask. The Wavelength code is smart enough to recognize and ignore this feature, so don't worry about it. There will be a local increase in the RMS error, but then my code enforces a smooth solution along the slit, so the local increase you show is not meaningful. Look at the rectified spectrum, does it look good?

~ N

On Wed, Mar 4, 2015 at 3:38 PM, hanaeinami notifications@github.com wrote:

I found a sudden increase of RMS near the center of one of the slits (slit 17) when I ran Wavelength.fit_lambda for my obsfiles. It appears that this is caused by a weird feature in the raw images (bad pixels?) that happens to lie on a strong sky emission line which is used for wavelength calibration.

I've attached a plot which shows the sudden jump of the RMS (vs. pixel and slit numbers) and a screenshot of a 2D image of the feature that caused this issue. The image is combined science images made with Wavelength.imcombine. This feature is extended around (x,y) = (266, 493). I think that because it happens to lie on a strong sky emission line, it messed up the wavelength fitting. I also found a similar feature at around (1320,1105), (1097, 575), (1008, 223).

These features appear in all of the data during our observing run in April 2014, and also in the data taken in September 2014. So I guess that these could have originated from the detector? So far, the wavelength calibration problem only happens when these features happen to overlap with a sky line, but if possible, it'd nice to be able to mask out this feature for the next version of the DRP.

Also, a minor bug report:

The slit numbers in the output of Wavelength.fit_lambda don't look right. For example, for the same mask above, Wavelength.fit_lambda says that:

S01 = pixels 7-77 S02 = pixels 1907-2031 S03 = pixels 1772-1894 S04 = pixels 1684-1762 ... S21 = pixels 221-298 S22 = pixels 132-210 S23 = pixels 88-122

On the other hand, Background.handle_background seems to handle the slit numbers correctly (has a -1 offset in the slit number, but that's OK):

slit 0 [2034,1904] slit 1 [1896,1769] slit 2 [1764,1681] slit 3 [1675,1593] slit 4 [1589,1505] ... slit 21 [124,85] slit 22 [79,4]

I've attached a screenshot below to visualize this. The lines and numbers at the top of the figure indicate the slit positions and slit numbers. The green ones are from the Wavelength.fit_lambda outputs and the orange are from Background.handle_background above.

Thank you, Hanae Inami

[image: s17] https://cloud.githubusercontent.com/assets/11051001/6495963/bbea7f96-c28b-11e4-8008-7be7934bd3b9.png

[image: 266_493] https://cloud.githubusercontent.com/assets/11051001/6495966/c035aff8-c28b-11e4-9aa8-f1bb9c6d70d6.png

[image: slit_no] https://cloud.githubusercontent.com/assets/11051001/6495969/c40664e2-c28b-11e4-97ad-03a8449c16fd.png

Reply to this email directly or view it on GitHub https://github.com/Mosfire-DataReductionPipeline/MosfireDRP/issues/17.

+1 831 704 6425

[hanaeinami]

Hi Nick,

Thank you for your reply. Sorry if this was already brought up before and I missed it.

Indeed the rectified spectrum looks fine, but the RMS were large, so it made me worry. So it sounds that I don't need to worry about large RMS if it was due to the volcanos.

Thanks! Hanae

[nickkonidaris]

Hanae: No need to worry. ~N

On Thu, Mar 5, 2015 at 9:21 AM, hanaeinami notifications@github.com wrote:

Hi Nick,

Thank you for your reply. Sorry if this was already brought up before and I missed it.

Indeed the rectified spectrum looks fine, but the RMS were large, so it made me worry. So it sounds that I don't need to worry about large RMS if it was due to the volcanos.

Thanks! Hanae

Reply to this email directly or view it on GitHub https://github.com/Mosfire-DataReductionPipeline/MosfireDRP/issues/17#issuecomment-77407315 .

+1 831 704 6425

[lucarizzi]

Hi Hanae I am trying to reproduce the other small bug that you mention, the one about the slit numbers and pixels being wrong, and I can't seem to find anywhere on the pipeline where such an output is produced.

You say that you get something like: S01 = pixels 7-77 S02 = pixels 1907-2031 S03 = pixels 1772-1894 S04 = pixels 1684-1762 ... S21 = pixels 221-298 S22 = pixels 132-210 S23 = pixels 88-122

But I when I run Wavelength.fit_lambda on my data, I don't get any output that looks like that.

Is there a chance that you might not be running the latest version of the pipeline?

[nickkonidaris]

Hi Luca, there's no bug. Hanae is talking about an increase in the outputted RMS at these pixel locations (she wrote her own code to parse the output of the drp). But the final RMS is much better because of a smooth wavelength solution enforcement criteria (that is weighted by the RMS). ~N

On Fri, Mar 6, 2015 at 11:25 PM, lucarizzi notifications@github.com wrote:

Hi Hanae I am trying to reproduce the other small bug that you mention, the one about the slit numbers and pixels being wrong, and I can't seem to find anywhere on the pipeline where such an output is produced.

You say that you get something like: S01 = pixels 7-77 S02 = pixels 1907-2031 S03 = pixels 1772-1894 S04 = pixels 1684-1762 ... S21 = pixels 221-298 S22 = pixels 132-210 S23 = pixels 88-122

But I when I run Wavelength.fit_lambda on my data, I don't get any output that looks like that.

Is there a chance that you might not be running the latest version of the pipeline?

Reply to this email directly or view it on GitHub https://github.com/Mosfire-DataReductionPipeline/MosfireDRP/issues/17#issuecomment-77677715 .

+1 831 704 6425

[lucarizzi]

Hi Nick Yes indeed but he mentions something else too, a small bug in the association between slit numbers and pixels and he includes a few lines of output from the pipeline to demonstrate it.

Look at his original message where it says "Also a small bug report".

Luca

On Mar 7, 2015, at 5:03 AM, Nick Konidaris notifications@github.com wrote:

Hi Luca, there's no bug. Hanae is talking about an increase in the outputted RMS at these pixel locations (she wrote her own code to parse the output of the drp). But the final RMS is much better because of a smooth wavelength solution enforcement criteria (that is weighted by the RMS). ~N

On Fri, Mar 6, 2015 at 11:25 PM, lucarizzi notifications@github.com wrote:

Hi Hanae I am trying to reproduce the other small bug that you mention, the one about the slit numbers and pixels being wrong, and I can't seem to find anywhere on the pipeline where such an output is produced.

You say that you get something like: S01 = pixels 7-77 S02 = pixels 1907-2031 S03 = pixels 1772-1894 S04 = pixels 1684-1762 ... S21 = pixels 221-298 S22 = pixels 132-210 S23 = pixels 88-122

But I when I run Wavelength.fit_lambda on my data, I don't get any output that looks like that.

Is there a chance that you might not be running the latest version of the pipeline?

Reply to this email directly or view it on GitHub https://github.com/Mosfire-DataReductionPipeline/MosfireDRP/issues/17#issuecomment-77677715 .

+1 831 704 6425 — Reply to this email directly or view it on GitHub.