Create recipe template for generating a BadPixelMap Calibration

[maxwellmb]

Is your feature request related to a problem? Please describe.

No recipe template exists for creating BadPixelMap from L1 data.

Describe the solution you'd like

A recipe template needs to be written to generate a BadPixelMap Calibration file from L1 data.

Additional context

In order to test this you will need to get your hands on the appropriate input data from II&T or TVAC and you will quantitatively compare the calibration file to the outputs of the II&T data pipeline. In addition please describe in your own words (~several sentences to a paragraph) what the recipe does and what the inputs and outputs are. Finally, please make sure the appropriate table entry is complete within the Confluence DRP Implementation Document table in Section 2.

Here's a checklist:

• [x] Recipe Written
• [x] Test Written and passing
• [x] Table Entry Complete

[juliamilton]

I've written the recipe and test file for this, but have a few outstanding questions:

• How should walker.py handle the master dark and master flat? I've added additional arguments to the walk_corgidrp function to accept these inputs for now, but I'd like to confirm if this approach is what you had in mind.
• The TVAC noise maps I have access to have multiple fields swapped between what the code expects to find in the primary and extension header, which causes several issues. Additionally, KGAIN is not defined in either of the headers for the TVAC images. I've made modifications to darks.py and the Image class to address the swapped header values, but I'm wondering if this is specific to the data files I have or if you’ve encountered similar issues with the datasets you’re working with.

[semaphoreP]

The master dark and master flat should be calibration files specified as part of the recipe json file. They should be specified as part of the bad pixel map creation step (see the other step functions as examples). You shouldn't be modifying the walk_corgidrp function.

I've not encountered such issues. The code I pointed out in the l1_to_l2b_e2e.py should work just fine to handle all of the calibration data you need. Have you gotten access to the box folder yet?

[juliamilton]

Got it, thanks. I reverted walk_corgidrp to its original arguments. For that, can I make the assumption that only one Dark and one FlatField calibration file will be in the caldb at a time?

Ok, good to know. It may just be an issue with the data I have then, since I did have to modify a lot to handle the noise maps. I don't have access to Box yet - since the invite was sent to my jpl.nasa.gov email address I have to get NASA approval for an account.

[semaphoreP]

If you set the calibration files to be chosen automatically in the recipe (with the "AUTOMATIC" value), then the caldb will automatically pick the most suitable calibration, even if there are multiple files in the caldb.

If we can share the Box folder with a different email, just DM me.

[juliamilton]

I just want to clarify: The bp_map test and functions shouldn't handle any input image frames, correct? My understanding is that the bad pixel map is generated solely from the flat field and noise maps.

If I'm interpreting the get_calib function which uses "AUTOMATIC" correctly, it uses the metadata from an input frame (like time and other attributes) to automatically select the appropriate calibration file. Since no input frames are being passed in this case, the automatic selection wouldn't work as intended. If that's the case, I could generate a dummy frame with the time and necessary metadata populated if using this function anyway is preferable.

I'll message you about the Box folder!

[semaphoreP]

You are correct. I think let's use a single dummy frame for now (e.g., another TVAC frame). We also shouldn't use the guess_template() functionality of the walker, and instead use the template keyword of walk_corgidrp() to specify that we should use the bad pixel map recipe template. So, the walker doesn't have to somehow guess to use the bad pixel map recipe from a random file.