Add a background keyword based on an estimate of it at Level2B/Level3

[stscijgbot-jp]

Issue JP-3222 was created on JIRA by Alberto Noriega-Crespo:

It is clear that there is a wide interest, practically & scientifically, on the trending of the  IR background in the JWST pipeline products. The MIRI instrument at long wavelength in particular is greatly affected by the Telescope/SC background. This ticket is to request the adding of a meaningful Background header keyword, that leverages on the current process that is being used to populate the BKGMETH and BGLEVEL header keyword in the Level 3 products ([https://jwst-pipeline.readthedocs.io/en/latest/jwst/skymatch/description.html).] 

One possibility could be to use the "local" method in skywatch that "simply computes the mean/median/mode/etc. value of the sky separately in each input image". But different approaches should be studied and validated.

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

Alberto Noriega-Crespo  To clarify this a little bit. Are you asking that a step be added in the level 2 pipelines to estimate a background level per image and output that in the cal (or equivalent) header? Since skymatch is run in the level3 pipeline (for some types of data), it's only added to the headers of the level3 pipelines in the combined images right now. I did a test and looked at a crf file, which is an output file on a per exposure basis (again, for some pipelines), and the keywords BKGMETH and BKGLEVEL are both in that file type.

Here's some information on the types of files that currently exist at the level 3 pipeline stages.

https://jwst-pipeline.readthedocs.io/en/latest/jwst/data_products/product_types.html#stage-3-data-products

Does this address your question? If this doesn't do it, can you clarify your question a bit further?

 

[stscijgbot-jp]

Comment by Alberto Noriega-Crespo on JIRA:

In level 2 would be the ideal place, since that would sample individually the backgrounds per tile. In level 3, unless one adds the backgrounds from each corresponding tile as a header keyword (not impossible but a 5x3 mosaic could have 15 values that could be added), that information gets lost or at best is an average. 

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

But this is the point I want to clarify: There is an extension called the crf file that is an output on a per exposure level (every file in the calwebb_image3 pipeline for instance) and that does have the two keywords. So I don't think my previous note was clear on this.

In a project I'm working, this is one of the output file headers: or at least portions of it:

FILENAME= 'jw01911001001_06101_00001_mirimage_a3001_crf.fits' / Name of the file

       Background information                                                                                                                                    BKGMETH = 'match   '           / Background computation method                   BKGLEVEL=  0.02120916127278258 / Computed/Matched constant background level      BKGSUB  =                    F / Has background been subtracted from data?      

 

This is an individual file, and part of the level 3 pipeline run. Now, this isn't one of the main files that people tend to examine, like a rate, cal or combined i2d file, but it does exist in MAST if you know to look for it. The question is, is this enough, or do you need to have it in a more frequently used file, like a cal file, which necessitates adding code to the level 2 pipeline?  

[stscijgbot-jp]

Comment by Alberto Noriega-Crespo on JIRA:

Thanks Misty. My understanding, based on reading the documentation, is that drizzle uses different approaches to match sky levels in the  overlapping regions. This OK, but we believe based on testing that   these values for BKGLEVEL are deltas, i.e. what one needs to apply to match the overall sky. What is needed is a true value for that product in MegaJansky per steradian. we suspect that this calculated prior to the "sky matching" (BKGMETH='match') done by drizzle.

I believe you got it with:  "do you need to have it in a more frequently used file, like a cal file, which necessitates adding code to the level 2 pipeline?  "

Code maybe is needed to add the values that are calculated, but I suspect that  the code is already determining a median or mean value for the Level 2b products and if so that is the value (no the current delta) that is needed.

 

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

Thank you. That does clarify things quite a bit for me. At this point, it sounds like we have enough to start discussions on the topic. I think we need a priority and a justification for this and we can proceed. How important is it that we add this keyword, and why is it so important? Those are things we'll need to know to get it properly put into the priority list of all the other work to do. Thanks.

[stscijgbot-jp]

Comment by Alberto Noriega-Crespo on JIRA:

Very valid point. We do need to see what other needs (perhaps more important) we have. In the long term there is the strong desire by us (MIRI) and the project (NASA Goddard) to monitor the background. Right now we (MIRI) have a 'work around' for the imager data based on the creation of FLATS using the science data, so on my mind  this is not critical and has a medium priority, but it should be done.

[stscijgbot-jp]

Comment by Anton Koekemoer on JIRA:

Misty Cracraft  Karl Gordon  (CalWG MIRI reps) would you be able to assign this ticket please the MIRI team's assessment of an "Impact" and "Urgency" rating (1 though 4) and associated text in the "Criticality Rationale" field?

[stscijgbot-jp]

Comment by Alberto Noriega-Crespo on JIRA:

Measuring the "background" can be used as an indictor of the health of the observatory (e.g. sunshield degradation or damage) and this can have a direct impact on the observations (e.g. higher and/or non-uniform background). MIRI is trending the background for the IMAGER data for all  its filters as requested by the project [and as indicated in my previous comment. The observers/users themselves, however, do not have a sense of the changes or variations themselves unless the background is registered. (or measured by them). The pipeline does an estimate of the background during the final steps of the Image reprocessing to create the Level 3 products [as mentioned above]. On my mind:

IMPACT= 3   URGENCY=2