Incorrect NIRISS gain reference files

[stscijgbot-jp]

Issue JP-2222 was created on JIRA by Michael Regan:

The NIRISS gain file does allow the correct calculation of the Poisson noise in the science data. The gain file needs not be corrected for IPC. The IPC correction is needed to calculate the total number of detected electrons but the post-IPC gain file will overestimate the Poisson noise. IPC causes a reduction in the observed Poisson noise thus using a gain file that has been corrected for IPC will yield an overestimate of the Poisson noise.

This will affect the noise calculation of the jump step and the weighting in the ramp_fitting step.

[hbushouse]

Closing this issue, because all of the work to be done is assigned to the INS MIRI team and does not involve any code changes.

[stscijgbot-jp]

Comment by Kevin Volk on JIRA:

There seems to be competing requirements here.  For the WFSS mode the desire is to work in electrons rather than ADU in the deblending of spectra, and the gain in that case needs to be corrected for IPC effects because the correction is different in the voids as opposed to regular pixels.  Using an uncorrected gain file would produce 2% jumps anytime a spectrum crosses a void edge.  Since there are many small voids in one channel of the NIRISS detector in addition to the large void, this would have a significant negative impact on WFSS science.  For the uncertainty calculation one does not want the IPC correction.  But of the two conflicting wishes, the WFSS mode has higher priority.  We do not want discontinuities in any spectrum that crosses the void areas in the output WFSS spectra.  The SOSS and AMI modes also want to work in electrons not in ADU.  Probably the AMI mode would not be much affected unless full frame observations are done.  For SOSS the small voids are a potential issue, although if only changes in signal are important then it may not be a problem.

Is this correct, or should one use the non-IPC corrected gain map for both cases?  It seems to me that the IPC-corrected gain map is needed the spectral modes, because the gain map is smooth(er) across the voids after the correction.

If we need the IPC corrected gain map for WFSS and the non-IPC corrected gain map for the jump and ramp fitting steps, then we would need to have two gain reference files in the pipeline.

 

[stscijgbot-jp]

Comment by Michael Regan on JIRA:

I guess there's a typo here this is not a MIRI issue. Has there been a non-IPC corrected gain file delivered?

[stscijgbot-jp]

Comment by Kevin Volk on JIRA:

We have not delivered a new gain reference file without the IPC correction because of the issue with the voids and the adverse effect it would have on the WFSS data.  I am also not convinced that the NIRISS gain is measured to such precision that this IPC/no IPC issue is the limiting factor in the noise calculation, because we see structure in the gain over the channel along the slow read direction for every data set from which we make a gain calculation, but I think that the gain should not be a function of the position within the channel.

 

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Reopening until folks come to agreement on whether updated NIRISS gain ref files are needed or not.

[stscijgbot-jp]

Comment by Kevin Volk on JIRA:

This was discussed with the full NIRISS team on 6 February.  The consensus was that the risks to WFSS decontamination outweigh the benefits.  The NIRISS team is willing to reconsider if it can be demonstrated quantitatively that an uncorrected gain file significantly improves the results and does not introduce artifacts that impact the data quality compared to the situation with our current reference file.