search

spacetelescope / jwst

Python library for science observations from the James Webb Space Telescope
https://jwst-pipeline.readthedocs.io/en/latest/
Other
569 stars 167 forks source link

Image3 output files for NIRISS imaging have distortions and stellar cores are rejected #5907

Closed stscijgbot-jp closed 3 years ago

stscijgbot-jp commented 3 years ago

Issue JP-2010 was created on JIRA by Swara Ravindranath:

Versions astropy: 4.2 pipeline: 1.1.0 asdf: 2.7.3 jwst: 1.1.0

In the i2d.fits output from Image3 step for the NIRISS imaging there are double stars which looks like the stars from the 4 images being combined are not aligned correctly. The outputs from the Image2 step have WCS that are aligned when displayed on DS9.

The files used for testing are uploaded on box:

https://stsci.box.com/s/r8rlzubdpsyny1tf7bvu9vyku3ml52qd

 

stscijgbot-jp commented 3 years ago

Comment by Swara Ravindranath on JIRA:

Kevin Volk

Can you please check if this is also an issue for the NIRISS imaging simulations? I am looking at the direct images for NIRISS WFSS. I see that the effect is more pronounced for F200W for which we shift the direct images significantly to cover the FOV.

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

I think I have seen such cases in simulations with saturated PSF cores, but it is not clear to me that these will be combined properly by drizzle.  In the non-saturated PSF cases I have looked at one does not see this type of issue.  I will run some further tests to see if I can reproduce this in an imaging simulation.

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

Swara, for some reason I cannot see the files in Box using the above link.

stscijgbot-jp commented 3 years ago

Comment by Swara Ravindranath on JIRA:

Kevin Volk

I have used the same simulations in the past and don't seem to have seen this issue. It would be nice to have another independent check as It could be that I am making some mistake.

I have shared the box link again through invite to the box folder. Please let me know if the files are accessible now.

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

I can see the files now.

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

I looked at the files and it appears that the same problem with the resampling when the stellar core is saturated that I saw before in a previous version of the pipeline is happening again.  The attached screen shot from DS9 shows part of the rate image for integration 1 on the left and the resampled i2d file for this individual image in the same region (I think) on the right.  One sees that the stars are badly resampled, which I think is due to drizzle having issues with the cores of the stars being saturated.  All the stars are getting badly resampled, and so the combined resampled image is even worse as it looks to me.

It is not clear to me what drizzle "should" do in such a circumstance.  Ideally the core saturated pixels should be ignored, but then it is not obvious how drizzle should assign values to these pixels and any ones that are adjacent and get some part of the signal from the core pixels.  I do not understand drizzle.

I think the situation here is worse than what I saw previously.  Maybe there has been some change in the drizzle default parameters for NIRISS that is making things worse.

 

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

There is something else besides saturated pixels affecting the results here.  The fainter of the two stars in the screen shot I put in does not have any saturated pixels in the core and yet it gets a bad resampling.  That seems to be a new thing.

I checked one of my pipeline testing simulations and that one does not seem to show this type of oddity in the resampled PSFs for the stars, all of which are arranged to be well below saturation.

stscijgbot-jp commented 3 years ago

Comment by Swara Ravindranath on JIRA:

Yes, its something new. I have added 2 png files. One from jwst0.16.1 back in July 2020. And the new one is jwst1.1.0. Sadly, I overwrote the files from some of the more recent runs between July 2020 and now.

stscijgbot-jp commented 3 years ago

Comment by Swara Ravindranath on JIRA:

Maybe I need to start with a fresh set of simulations with more recent MIrage updates.

 

stscijgbot-jp commented 3 years ago

Comment by James Davies on JIRA:

If I turn off tweakreg via:

$ strun calwebb_image3 image3_asn.json --steps.tweakreg.skip=True

the problem goes away, and the resampled results look good. When turned on, tweakreg source finder is finding 100 sources in each image (there are actually 14 according to Swara), and it is trying to find a new WCS solution based on bogus noise.

So I think this is a problem of the default parameters of tweakreg not being optimal for this simulated NIRISS imaging data. Clearly it would be good to have a pars-tweakregstep reffile in CRDS for NIRISS.

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

Why is tweakreg being used in the resampling of a single image?  Or am I misunderstanding?

stscijgbot-jp commented 3 years ago

Comment by Howard Bushouse on JIRA:

Ask and you shall receive. pars-tweakregstep param reffiles have recently been installed in CRDS for NIRISS (and other instruments). So now it's a question of whether the param values in those reffiles need any updating.

stscijgbot-jp commented 3 years ago

Comment by James Davies on JIRA:

The following tweakreg parameters seem to produce decent output:

$ strun calwebb_image3 image3_asn.json --steps.tweakreg.kernel_fwhm=4 --steps.tweakreg.snr_threshold=15 --steps.tweakreg.fitgeometry=rshift --steps.tweakreg.expand_refcat=True --steps.tweakreg.minobj=5

Certainly not optimized, but at least the noise in the input images doesn't seem to be throwing off the WCS fit.

stscijgbot-jp commented 3 years ago

Comment by Howard Bushouse on JIRA:

Kevin Volk the image3_asn file that was provided contains 4 exposures.

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

I had thought that tweakreg is calculating the image to image transformations and not affecting the resampling for an individual image from CAL to I2D.  Apparently this is not correct.

stscijgbot-jp commented 3 years ago

Comment by Howard Bushouse on JIRA:

tweakreg updates the WCS of each individual image in order to get them to all supposedly match one another. So if the updated WCS for one of more images is incorrect, the resample/drizzle step will lay down signal from those images into the wrong place in the combined/resampled image. The output that's being produced in this test case is a combined image, even though the file name looks like that of an individual image.

stscijgbot-jp commented 3 years ago

Comment by James Davies on JIRA:

One more thing I'll note here is that the NIRISS pars-tweakregstep reffiles delivered have kernel_fwhm set to something smaller than the 2.5 pixels used as the default for all instruments in the code. Depending on filter, the NIRISS reffiles have it as being between 0.569 to 2.462 pixels. Which seems much too small for a smoothing kernel that is supposed to help distinguish point sources from outliers. I.e. that change is in the wrong direction.

I would expect kernel_fwhm to range between 3 and 5 pixels or more, depending on the effective FWHM of the average light coming through a given filter. This change alone may account for most of the problem seen here. I used 4 pixels above to get decent results, but that is certainly not optimized.

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

The values for the kernel_fwhm are the expected values for the NIRISS filters from WebbPSF models.  The PSF is rather badly undersampled for the short wavelength filters in particular.  If the kernel_fwhm values we need to provide are not supposed to be a reasonable estimate of the actual FWHM in the point source profile, then we need some help as to what exactly the value should be.

 

stscijgbot-jp commented 3 years ago

Comment by James Davies on JIRA:

kernel_fwhm is the size in pixels of the gaussian kernel that is run over the image to aid in source detection, i.e. to optimize SNR of the detected sources against the background noise

https://github.com/spacetelescope/jwst/blob/8e029b69a32fd675cda99204323363b0588933c9/jwst/tweakreg/tweakreg_catalog.py#L21-L25

using DAOStarFinder from photutils:

https://github.com/spacetelescope/jwst/blob/8e029b69a32fd675cda99204323363b0588933c9/jwst/tweakreg/tweakreg_catalog.py#L70-L73

Only the positions of these detected sources are used in tweakreg when it fits a new WCS, so it's important that it detect only sources, not noise.

I dont' know how this relates directly to the WebbPSF models, but it's probably a function of the intrinsic PSF at a given wavelength, the sampling, and probably needs to be figured out empirically. I suspect for the filters where the detector undersamples the PSF, the difference between sources and jumps not removed by jump, ramp_fit is not very clear to the finder. Maybe Larry Bradley might know more?

stscijgbot-jp commented 3 years ago

Comment by Howard Bushouse on JIRA:

Could this be an issue with the PSF's in the simulated data not being a good match to what was assumed when choosing the values for kernel_fwhm in the tweakreg param ref files?

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

The PSFs for Mirage and the ones used to estimate the PSF FWHM all come from WebbPSF so there should not be any large discrepancy.  And the documentation does state that the kernel_fwhm value is the FWHM value for the Gaussian kernel image.  Presumably the kernel image size is larger than the FWHM value by some factor of order a few.  I do not know whether it is a good idea to have a Gaussian FWHM of 4 pixels when the expected intrinsic FWHM is much smaller than this for all our filters.  Some experimentation will be needed to see what the situation is.

 

stscijgbot-jp commented 3 years ago

Comment by Swara Ravindranath on JIRA:

Source detection algorithms usually require a smoothing kernel to detect sources above the noise. For HST images we used SExtractor for finding sources that were needed to do the tweakreg in multidrizzle, and that code uses a smoothing kernel as well. When I run the JWST pipeline source catalog step I manually set the kernel FWHM to 3.0, and npixels to 10. Those criteria may not necessarily reflect the true properties of the PSF, but are useful for the source detection algorithm. The smoothing kernel is only used as a low-pass filter to reduce the noise and help with the detection algorithm to identify pixels above the threshold. It is not used for the actual photometric measurements at least in SExtractor, and that is probably the case with photutils as well.

stscijgbot-jp commented 3 years ago

Comment by Kevin Volk on JIRA:

Very well, if this kernel is not for fitting the PSF then there need not be any particular relation between the value used and the actual PSF FWHM.  We will need to revise the parameter reference file with better values.

 

stscijgbot-jp commented 3 years ago

Comment by James Davies on JIRA:

A further note. These parameter reference files need only have the parameters in them that need changing based on the instrument and any CRDS selectors for that instrument. If the default in the code is fine, don't include it in the parameter reffiles.

So as an example for tweakreg, the searchrad parameter is probably not instrument-dependent. If there's a mean offset of N arcsecs in pointing for JWST, it's going to manifest itself the same for every instrument. So this should be set for the observatory and not set in any pars-tweakregstep reffiles. And the easiest way to do that is to use the default in the python code, and if that needs an update during or after commissioning, update it there, instead of having to redeliver dozens of new reffiles for each instrument. That's busy work we don't need.

So my suggestion would be to redeliver these reffiles for NIRISS and only have the following pars defined:

class name kernel_fwhm

The first two are required for these files to work at all. kernel_fwhm is the only thing currently in the NIRISS pars files that is different from the defaults. Also include any other parameters that are different from the default that are needed to empirically make tweakreg work for NIRISS data based on the CRDS selectors, i.e. FILTER, etc.

stscijgbot-jp commented 3 years ago

Comment by Swara Ravindranath on JIRA:

Kevin Volk

I agree we need to revise the parameter reference files with larger kernel_fwhm. By using very small values we are probably using a sharpening kernel rather than a smoothing kernel.

stscijgbot-jp commented 3 years ago

Comment by James Davies on JIRA:

Since there's no work to be done here in jwst code, and it is a CRDS reffile issue, I will close this JP issue. I've linked this ticket to the appropriate CRDS issue dealing with these pars-tweakregstep reffiles.