Resampling edge effects for off-center MOS targets

[stscijgbot-jp]

Issue JP-3365 was created on JIRA by Melanie Clarke:

In performing calibration analysis for MOS mode, we discovered that there are discontinuities in the extracted level 2 spectra and s2d images at the upper and/or lower nod positions. These may be as large as ~ 40%.  The cause appears to be that there is missing flux along partial rows of the upper and lower edges in the resampled s2d file.  These edges correspond to areas in the cal file set to NaN due to missing WCS information.

The effects in level 3 products appear to be much smaller and/or absent, since combining the nods results in at least some coverage for the full spatial extent.  However, flux calibration and/or data quality may still be impacted in ways that could be improved if the wings of the upper and lower spectra were included in the final combination.

We can currently work around this issue by editing the MSA metadata FITS files to add an extra "open" shutter to each defined slitlet.  That is, for each 3-shutter slitlet, we pretend that 5 shutters were open, where the actual nod pattern uses only the central 3.  This allows the wings of the spectra to be included in the cal file and resampled properly in the s2d file, prior to extraction.  However, this workaround may also impact calibration for the source, including the pathloss correction (JP-3357).

The NIRSpec team will continue analyzing the issue to determine an appropriate fix.  We may want to extend the WCS coverage to all pixels currently included in the cal file.  We may also want add an extra pixel or two to the cutout boundaries to further mitigate the edge effects.

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

I have provided a test case for JP-2256, showing poor resampling results with IVM weights, that also demonstrates the discontinuities in nodded spectra.

Test data and notebook are at: ████████████████████████████████████████████████████

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

It looks like this issue should have an easy fix.

In the assign_wcs step, there is a hard-coded margin added to the MSA slit y limits, around line 618 of jwst/assign_wcs/nirspec.py, in the get_open_msa_slits function.  It is set to 0.05, in slit units.  Increasing it to 0.5 has exactly the effect we want: the WCS is extended a couple pixels at the top and bottom of the slitlet.  The source position in the slit is unchanged, so pathloss/wavecorr corrections are not affected.   I will attach some plots from the test data set mentioned above, from PID 1493: 3 nods with and without the extra margin, resampled with exptime weights.  The extra margin fixes the discontinuities in the slit and makes the nodded spectra align much better.

It is possible with this change that some light contamination from adjacent open slits may appear at the edges of the slit cutouts.  This should not affect default extractions for extended sources, which are set to extract a fixed width at the center of the slit.  For point sources near the center of the slit, there should be no impact.  For point sources near the edges, we believe this change is an improvement over the current situation, where flux is lost due to edge effects.  User-defined extractions from the full slit, or near the edges of the slit, may be impacted.

I have made this small change in a branch on my fork of the jwst pipeline. It can be installed for testing with:

pip install git+https://github.com/melanieclarke/jwst.git@mos_cutout_margin

Charles Proffitt has tested his MOS calibration data set with the change and reports that the discontinuities no longer appear and that the consistency between nods is improved over the workaround padding the slit with extra shutters.

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

The NIRSpec team is happy with this solution, so I have included the changes in a PR here: #7976

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Fixed by #7976

If SCSB makes another release candidate yet for B10.0, we'll include this. Otherwise it'll get relegated to B10.1.

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

Tested with v1.12.2 and working as expected.

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

Closing: fix is working as expected.