spacetelescope / jwst

Python library for science observations from the James Webb Space Telescope
https://jwst-pipeline.readthedocs.io/en/latest/
Other
558 stars 164 forks source link

Background subtraction of 5 point nods in S1600A1 subtracts too much source flux #8301

Open stscijgbot-jp opened 6 months ago

stscijgbot-jp commented 6 months ago

Issue JP-3551 was created on JIRA by Charles Proffitt:

For the five point dither pattern in the NIRSpec S1600A1 in FS mode, the primary dither spacing is only 0.2571", (with some additional variation if spatial sub-dithers are also included). This leads to significant overlap of the PSF wings of point sources in each image with the wings of the PSF in the background images and can reduce the net extracted flux by about 25% relative to the baseline calibration when the usual nod subtraction is done.  But since we have 5 dither positions, this could be significantly reduced by not including the one or two immediately adjacent dither positions as background members in the level 2 files. One way to do this would be to require a minimum separation in the cross-dispersion direction before allowing another association member to be used as a background member for a given science exposure.  Taking into account the additional sub-pixel dither offsets, and the dither patterns used for other slits, I believe that a requirement that the cross dispersion position of background members be offset by no less than 0.375" would have the desired effect. For my tests, I used the YOFFSET keyword to determine the offset between the dithers, but this might not be a general solution as one doesn't want to impact the use of a dedicated background observation at a different pointing. This should perhaps be a tunable parameter in a reference file in case other instruments or setups would benefit from a similar restriction. Even if we eventually come up with an improved scheme for correcting for the flux changes resulting from nod overlap in general, not subtracting such closely spaced members from each other should still improve data quality as subtracting the overlap and then adding back in an approximate correction would add noise, and so this fix would remain in place.

A workaround is available, as the user can always edit the level 2 association files to exclude the nearest background members and then re-run spec2 and spec3, but a large fraction of the NIRSpec fixed slit calibration observations taken of standard stars use the S1600A1 with a 5 position dither and it is a bit awkward for the pipeline to give default reductions that are so far off in flux for standard stars.  If nothing else it has generated a number of help desk questions.

The rate files from Obs 160 and 107 of PID 1538 should provide good examples, as this star is bright enough that there is no significant real sky background.

stscijgbot-jp commented 3 months ago

Comment by David Law on JIRA:

Charles Proffitt Following up on this, a potential easier option than modifying the association generator might be to tweak the parameters of the bkg_subtract step instead.  The four other nod positions all get combined together using a sigma-clipped mean, and it looks like the default 3-sigma clipping isn't doing a great job in the case of the short S1600A1 slit at the longest wavelengths (i.e., largest PSF).  If I change it to use 1.5-sigma clipping instead though (steps.bkg_subtract.sigma) it seems to do dramatically better at eliminating the overlapping source flux from the composite background.

Might this be an option for you instead?  This way the only changes necessary would be for the NIRSpec team to deliver new pars-backgroundstep parameter reference files for whichever slits were problematic, and could adjust the sigma-clipping as necessary.

stscijgbot-jp commented 3 months ago

Comment by Charles Proffitt on JIRA:

Relying on the sigma clipping to decide which flux to include or not include would probably make it impossible to get a reliable absolute calibration as the rejection would depend a lot on the level of the source signal and errors. In addition, I'd think it would do bad things to the S/N as there would always be some points on the edge of rejection or acceptance and that would vary from pixel to pixel.   So my initial reaction is that this is probably not a good solution. We need the flux calibration to be repeatable for sources with a wide variety of flux levels and SEDs.  For the fixed slits better than 1% should be achievable, although this will also require some additional corrections.

stscijgbot-jp commented 3 months ago

Comment by Melanie Clarke on JIRA:

You could still reject background members with a check on X/YOFFSET in the step, instead of modifying the association generator.  That would make it easier to allow the minimum offset to be a tunable parameter.

stscijgbot-jp commented 3 months ago

Comment by Christian Hayes on JIRA:

Melanie Clarke from the pipeline perspective, would addressing this in the step be preferred to modifying the association generation?  As Charles Proffitt points out we would be concerned with using the sigma clipping to remove nearby background flux because it will depend on S/N, but we would be fine with removing nearby nods in the background subtraction step (with logging to make it clear what is being done) if that would be preferred.  

stscijgbot-jp commented 3 months ago

Comment by Melanie Clarke on JIRA:

I think it just depends on what you expect will be most useful to the user.  If it's better to just never see the closest dithers as background candidates, we could fix it in the association file.  If you think the user might sometimes want to consider them as background candidates, and decide based on the distance from the source whether to use them, then that's better implemented in the pipeline, so that a dynamic check can be done and the user can change the parameters if needed without making new association files.  

If we're not totally sure yet what the optimal distance is, it might be better to implement it in the pipeline first, so that we can experiment with it.  If we find a hard and fast rule, we could implement it in the association files later.

stscijgbot-jp commented 2 months ago

Comment by Charles Proffitt on JIRA:

For the S1600A1, there really isn't much choice.  The close spacing in the 5 point dither is too close to give good results.  But for the standard 3 point dither in the S1600A1 there even less choice as there isn't any redundancy in the spacing for the central nod position; you have to use the close spacing there if you want to do background subtraction at all.  So all the ability to adjust the threshold would do might be to allow some selections on the margins when spatial subsampling is selected and creates addition spacings. Where it gets more complicated is for the other slits.  Due to the longer slits and wider nod spacing, it isn't unreasonable to use the closer spacings as background for those slits. But perhaps excluding them might give better results in some cases?  That still needs to be investigated in detail, although we don't really have a lot of data to test that with.  The user can always rewrite the level2 association file manually to select the desired background pairings. That would require a bit more work than simply setting a threshold, but might be more transparent than having background members in the association which aren't used as background because they don't meet the threshold.

stscijgbot-jp commented 2 weeks ago

Comment by Melanie Clarke on JIRA:

Christian Hayes Charles Proffitt - do you have a recommended path forward for this issue at this point, or does this still need more study?

stscijgbot-jp commented 2 weeks ago

Comment by Charles Proffitt on JIRA:

Sorry, I was thinking of the ticket that applies the correction.  In practice whether we're applying a correction or not, dithers with too small a spacing should be omitted as background members, especially when there are other dithers that can be used as background as is the case here.  So I don't see any reason not to move forward with this one.

-In practice it needs more study. In particular the corrections for the narrower FS and MOS will be more complicated, although perhaps we could initially implement for just the S1600A1 where the effect has the biggest impact.-

stscijgbot-jp commented 2 weeks ago

Comment by Melanie Clarke on JIRA:

Okay, sounds like this is ready for development then: we should modify association files for NIRSpec FS S1600A1 data with 5 dithers to exclude background members with offsets < 0.375" from the science member.

I don't think there is any way to provide a reference file to the association generator to make that value tunable, so it will be hard coded.  If the number needs to change, it will require a software update.

I assume no other slits need handling at this point, and that we should not exclude the background members for any other dither pattern for S1600A1.

Charles Proffitt - is that right?

David Law - any other thoughts?

stscijgbot-jp commented 2 weeks ago

Comment by Charles Proffitt on JIRA:

Yes, it's only the 5_SPECTRAL, 5_SPATIAL, and 5_BOTH dither patterns in the S1600A1 that are affected.  The 0.35" value was selected to limit the close dithers in these cases without affecting other patterns or other slits even if the same rule was applied to them. I do presume that when selecting a sub-dither pattern, that the software is already smart enough not to use the small spatial or spectral sub-dithers around the same primary point as backgrounds for each other as that would give nearly 100% overlap! But perhaps that's done at a higher level in the system.

stscijgbot-jp commented 2 weeks ago

Comment by David Law on JIRA:

Making this change to the association file generator sounds reasonable for the 5-point case, although we should be sure that the system doesn't crash if it is unable to find any viable backgrounds (e.g., perhaps one or more exposures in the sequence were missed).  Failure case would presumably be to include no backgrounds, and thus not perform background subtraction.

Question for Charles Proffitt on the 3-dither case.  As stated above, this effectively cannot be properly calibrated as there is not enough separation between the dither positions.  Is it best for such cases to perform no background subtraction, or to make no changes to how background subtraction is currently done (knowing that the result will be bad)?  Is 3-point dither in S1600A1 allowed in APT?  If so there should be a note on the NIRSpec FS JDox page warning users not to use this combination as the results cannot be calibrated properly.

stscijgbot-jp commented 2 weeks ago

Comment by Charles Proffitt on JIRA:

For the 3 point nod in the S1600A1, the nominal separation of the pointings is 0.4556" rather than the 0.2571" of the 5 point dither. For the default 5 pixel (0.5") extraction box, this reduces the overlap from ~ 50% to less than 10%, which gets diluted a bit more in the final average over all three dithers.  So not great, but the resulting total errors are much more modest that with the 5 dither case. Ultimately we'll want to apply a correction such as is envisaged in JP-3647, but even with that correction, we don't want to include the very close dithers with the very large overlap as background members as even with a correction we'd still be adding substantial noise..

stscijgbot-jp commented 2 weeks ago

Comment by David Law on JIRA:

Charles Proffitt That sounds like no change for now in the 3-dither case then, but nonetheless it would be helpful to add something to JDox warning users about this problem (and perhaps suggesting use of a different pattern).

stscijgbot-jp commented 2 weeks ago

Comment by Melanie Clarke on JIRA:

I have a proposed solution for this issue here:

8744

Currently the association generator does not have access to spatial offset information like YOFFSET, but it does have the dither position number for each exposure.  From JDox, it sounds to me like the 5 point dithers always use a fixed sequence of positions, so it should be sufficient to exclude background members based on the pattern index (PATT_NUM). 

My proposed solution just checks the pattern number for each background candidate and excludes it if it's not more than 1 away from the science member. So, if science is position 1, background at position 2 is excluded. If science is at position 3, backgrounds at position 2 and 4 are excluded.  This applies only to NRS_FIXEDSLIT, with S1600A1, if there are 5 dither points.

Testing with PID 1538 obs 160 and 107, it looks like this works well for these observations, and the output spectra are much improved.

Charles Proffitt - is there any reason this solution might not generalize well? Are there any known exceptions to the dither pattern order?  Or any other reason the matching really must use the YOFFSET information?

stscijgbot-jp commented 2 weeks ago

Comment by Charles Proffitt on JIRA:

Actually there are also  5_SPATIAL, 5_SPECTRAL, and 5_BOTH dither patterns with up to 20 dither points, some of which are close spaced and some not.  When doing the sub-pixel dithers, I think the system somehow the system already knows not to use the sub-pixel dithers around the same primary position as backgrounds, even though the PRD table .  Since the spectral dithers add shifts in the dispersion direction only, this has to be the case, otherwise it would be adding backgrounds that contain 100% of the source flux. I'm attaching the SOCPRD dither table for this aperture. 

Presumably somewhere in the system, perhaps in APT, it is recognized that the "5" point 5_BOTH dither pattern with 20 dither positions consists of 5 primary dither positions each of which has 4 sub-dithers, and so the association rules know that sub-dithers within the same primary position can't be used as background members for each other. So for all of the "5" point dither patterns in the S1600A1, the rules should be changed to also avoid allowing any of the sub dithers in the immediately adjacent primary dither sets from being allowed as backgrounds. I'd think it would be best to do this when building the level 2 asn file, rather than adding a rule to spec2 to ignore already included background members.

 

stscijgbot-jp commented 1 week ago

Comment by Melanie Clarke on JIRA:

Okay, thanks Charles.  I'll look for examples of those dither patterns and see how their association files are handled.

stscijgbot-jp commented 1 week ago

Comment by Christian Hayes on JIRA:

Melanie Clarke for the sub-pixel dithered data, PATT_NUM increments with each sub-pixel dither, so I don't think this would generalize well for those.  I don't think there are any such data taken yet, but it might be good to have a general solution to catch any future cases.  But I believe the dither patterns are fixed, so it should be possible to back out the primary dither number and do the same check on that.  I can suggest a solution on the PR.

stscijgbot-jp commented 1 week ago

Comment by Charles Proffitt on JIRA:

There probably aren't a lot of examples in the S1600A1.  PID 1492 obs 5 did 2_SPECTRAL in the S1600A1, and obs 1 did 5_SPECTRAL in the S200A1, so those should at least illustrate how the association files are constructed. Also, PID 3522 used 5_BOTH in the S200A1 and the S200A2 in separate observations.

stscijgbot-jp commented 1 week ago

Comment by Melanie Clarke on JIRA:

Excellent, thanks both!