search

spacetelescope / jwst

Python library for science observations from the James Webb Space Telescope
https://jwst-pipeline.readthedocs.io/en/latest/
Other
558 stars 164 forks source link

EMICORR for MIRI LRS slitless does not work for ngroups < 10 #8427

Open stscijgbot-jp opened 4 months ago

stscijgbot-jp commented 4 months ago

Issue JP-3597 was created on JIRA by Andreea Petric:

The emircorr step aims to remove the 390Hz noise pattern in rate and rateints calibrated products, as that pattern is prominent in rate images with short integrations. We ran the JWST pipeline version 1.13.4 and 1.14.0 as well as the development version 1.14.1.dev18+g9429dc65. with crds jwst_1223 and jwst_1225 pmas and found that the noise structure was still present. However, because the faint and regular 390Hz noise structures may not have a dramatic limiting impact on science, we label this as medium critical. 

stscijgbot-jp commented 4 months ago

Comment by David Law on JIRA:

Adding a note that Michael Regan and Eddie Bergeron have been looking into this and found that the issue is specifically with data with exactly 7 groups (with an 8th reset group), for which the 390 Hz noise is in phase and cannot be removed by the current approach.  Discussion ongoing within the MIRI team on what the best path forward is.

stscijgbot-jp commented 2 months ago

Comment by Ian Wong on JIRA:

I have tested the emicorr step on several commissioning LRS slitless datasets with a range of integration lengths. I've found that the EMI correction does not work well whenever the grp/int is less than ~10. There are a few sources of the poor performance:

  1. The sloper() function within the step, which does a quick and dirty slope fitting for groups 1:ngroups-1, uses an analytic linear fit algorithm that is not optimized for small numbers of points. For 5 grps/int, it only has three ramp points to work with, and it'll exactly pass through the second and fourth points. After the routine removes the slope to flatten the ramp, those two frames are now identical. For slightly longer integrations, this step is still unreliable, as the individual ramp points are themselves affected by the noise, which can bias the slope fit.

  2. The minmed() function is used to calculate a "superbias" to remove common-mode flux from the flattened ramps. The default calculation is a simple median of the frames. In the case of 5 grps/int, the superbias ends up being identical to the second and fourth frames, and they end up being completely zero, severely throwing off the subsequent phased amplitude fit.

stscijgbot-jp commented 2 months ago

Comment by David Law on JIRA:

Edited the ticket title to reflect that this is a broader issue affecting ngroups <~ 10 rather than just 7 groups (although 7 groups may still be a special case).