New SINKCORR for CALACS

[pllim]

CALACS wants to do SINK pixel flagging like CALWF3 but might use a slightly different scheme of flagging.

Currently not sure if this should be done by Jamie as part of #13 (new PCTECORR) or in parallel by me (in case Jamie does not have time). I think Megan did both at the same time.

Implementation Details

Following details are extracted mostly from links below, and some from private communications:

• New reference files with suffix *_snk.fits with DQ values 1024 for affected pixels. This file will be populated by CRDS into SNKCFILE FITS header keyword in the PRIMARY header of RAW image.
• New flag named SINKCORR in the PRIMARY header of RAW image, stating whether to apply SNKCFILE or not.
• Should only be applied if BIASCORR=COMPLETE.
• Format: The file is a two-dimensional FITS MEF but does not contain ERR or DQ image extensions. There are 2 SCI image extensions, Chip 2 in EXT=1 (CD) and Chip 1 in EXT=2 (AB), in RAW fullframe unbinned size of 2068 x 4144. Storage wise, wouldn't it be more efficient to store this info as a binary table? Ah, well.
• Test SNKCFILE was provided by Jenna Ryon.

Questions:

• [x] Why are there no DQ extensions in reference image? It is much easier for INS to calculate however they want to populate the DQ and just deliver the DQ values to populate. It is inefficient for CALACS to recalculate the same thing over and over again based on SNKCFILE's SCI data for every input image. The ISR talks about flagging based on background level but how is CALACS supposed to estimate background blindly? Need science image pixel value to calculate (not global background). Exact flags will be different for each images.
• [x] Will all WFC images have these new keywords? That is, full reprocessing of static archive. It is mentioned that there "will not be a SNKCFILE before January 2015," but does that mean no keywords or they are set to OMIT and N/A? Yes.
• [x] Where exactly in CALACS is this correction step going? After BLEVCORR, before PCTECORR, which puts it as the last step in ACSCCD.
• [x] If it is set to PERFORM but BIASCORR!=COMPLETE, skip silently, error, or warning? Just warning.
• [x] Extract subarray region for subarrays? It is implied but I want to make sure. Yes.
• [x] Which pieces of CALWF3 do you want to copy over? wf3ccd/sink.c

Todo:

• [x] Need to implement a new step for this.
• [x] Need Jenna's Python flagging script for reference.
• [x] Need test images and SNKCFILE for regression test. Maybe a fullframe and a subarray. That includes science input, reference file, and expected output.

Update from Norman (2017-02-14)

This has nothing to do with the new CTE code. The intent is to clone the CALWF3 sinkcorr stage into CALACS, with minor modification due to our change in reference file conventions.

We will generate a new sinkcorr reference file every anneal cycle (but only in the flashed-darks era), with appropriately indexed USEAFTER. We also change the head-pixel labeling convention, to reflect the depth of the sink rather than its birthday. We may use a different DQ bit for WFC sinkpix than is used by CALWF3: Jenna and/or David can confirm.

Jenna should be able to walk you through the modifications to the CALWF3 sinkcorr code (perhaps with a little assistance from David)? She should be able to hand you a prototype WFC sinkcorr reference file immediately.

Related Links

• Sink Pixels in ACS/WFC (draft) by Ryon & Grogin 2017, Section 6
• Problem Report 86721 (New Sink Pixel Flagging Reference File)
• Problem Report 86722 (New Keyword for ACS/WFC Sink Pixel Flagging)
• New Reference File Type for ACS/WFC Sink Pixel Flagging

[dborncamp]

I think Norman envisioned that it would be very similar to the calibration that WFC3 has implemented so much of the work could be cribbed from there. I think the relevant WFC3 module is: https://github.com/spacetelescope/hstcal/blob/master/pkg/wfc3/calwf3/wf3ccd/sink.c .

I think this should happen after DARKCORR, but maybe Jenna has a better idea of where it should be.

I am not sure how to handle this with the old subarray read out mode as these pixels may behave differently (have we looked into it?), maybe only support full frame?

Should notify the user if BIASCORR is not on. But just warn, it is not that big of a show stopper.

@pllim Which anneals do you need the file for to do regression testing?

[jryon]

Norman helped us with the remaining questions.

This should happen after bias and blev correction but before CTE correction. This is because the reference images are created from non-CTE corrected darks.

As for subarray data, this should just extract the subarray region from the reference image and apply the correction as you would to full-frame data.

[pllim]

@jryon , what does "downstream" mean in your Section 6? Is it the pixel closer or farther from the amplifier?

[pllim]

@jryon , are there any overlapping sink pixels? Or are they all guaranteed to be separated from each other with zero-pixels in between?

[jryon]

@pllim Downstream means closer to the amplifier. There shouldn't be any adjacent sink pixels, but you can have a sink pixel overlap the trail of another sink pixel in the same column. They aren't guaranteed to be separated by zero-value pixels.