Update WFPC2 header Keyword 'APERTURE'

[stscijgbot-hstdp]

Issue HLA-1110 was created on JIRA by Jennifer Mack:

SVM drizzled products have a keyword added named 'APERTURE', which has been set to 'WFPC2'.

We would like to replace this with the values from DADs using the database keyword 'sci_aper_1234', as recommended by Lisa Sherbert . 

For example hst_6192_b1wfpc2{}wfpc2{}_f814w_u2xgb1_drz.fits would have APERTURE='WF4'

---

Before making this change, 1.) check that changing the keyword string does not affect the output filenames. 2.) check that it does not affect the selection of the JSON files:

https://github.com/spacetelescope/drizzlepac/tree/master/drizzlepac/pars/hap_pars/svm_parameters/wfpc2/wfpc2 ](https://github.com/spacetelescope/drizzlepac/tree/master/drizzlepac/pars/hap_pars/svm_parameters/wfpc2/wfpc2)e.g. wfpc2_wfpc2_astrodrizzle_any_n2.json

[stscijgbot-hstdp]

Comment by Michele De La Pena on JIRA:

Please read any Slack channel discussion regarding this ticket.  It is preferable that our software NOT read the DADs database, so we need to come up with a way (if possible) to get the value of the database value 'sci_aper_1234' some other way.  I think we will also want to draw on Warren's opinion for this.  Drizzlepac already communicates with the astrometry database, but minimizing the communication channels to external sources from the software which can cause potential exceptions if the preferred situation.  Also, external users employing the software will more than likely not be able to perform this communication.

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

Hi Michele,  I believe we can do this another way by taking keywords from the u*_shm.fits file. Lisa had mentioned that the sci_aper_1234 value in DADs is based on a combination of these 4 keywords so we could do the same.

APER_1  = 'WF4            '    / instrument aperture                             APER_2  = '               '    / instrument aperture                             APER_3  = '               '    / instrument aperture                             APER_4  = '               '    / instrument aperture        

I'm not sure if we always need all four or if the info is captured in APER_1.  She had showed an example of a ramp filter where the combined value was 'WF2-FIX;FR680P15' 

[stscijgbot-hstdp]

Comment by Lisa Sherbert on JIRA:

The small amount of data we have in AUTO happens to have single sci_aper_1234 values instead of multiples separated by a ';'.  Here are my poller file contents for that test:

# ../pipe/cache/ingest/SVM/20231002/wfpc2_231_1s/wfpc2_231_1s_input.out
u2311s03t_flt.fits,WF2
u2311s04t_flt.fits,WF2

# ../pipe/cache/ingest/SVM/20231002/wfpc2_2ls_02/wfpc2_2ls_02_input.out
u2ls0201m_flt.fits,WF4-FIX
u2ls0202m_flt.fits,WF4-FIX
u2ls0203m_flt.fits,WF4-FIX
u2ls0204m_flt.fits,WF4-FIX
u2ls0205m_flt.fits,WF4-FIX
u2ls0206m_flt.fits,WF4-FIX

# ../pipe/cache/ingest/SVM/20231002/wfpc2_2s0_02/wfpc2_2s0_02_input.out
u2s00201t_flt.fits,WF3-FIX
u2s00202t_flt.fits,WF3-FIX

# ../pipe/cache/ingest/SVM/20231002/wfpc2_350_17/wfpc2_350_17_input.out
u3501701t_flt.fits,PC1-FIX
u3501702t_flt.fits,PC1-FIX
u3501703t_flt.fits,PC1-FIX
u3501704t_flt.fits,PC1-FIX
u3501705t_flt.fits,PC1-FIX
u3501706t_flt.fits,PC1-FIX

# ../pipe/cache/ingest/SVM/20231002/wfpc2_3i9_04/wfpc2_3i9_04_input.out
u3i90401t_flt.fits,PC1
u3i90402t_flt.fits,PC1
u3i90403t_flt.fits,PC1
u3i90404t_flt.fits,PC1
u3i90405t_flt.fits,WF3
u3i90406t_flt.fits,WF3
u3i90407t_flt.fits,WF3
u3i90408t_flt.fits,WF3
u3i90409t_flt.fits,WF4
u3i9040at_flt.fits,WF4
u3i9040bt_flt.fits,WF2
u3i9040ct_flt.fits,WF2

# ../pipe/cache/ingest/SVM/20231002/wfpc2_482_c7/wfpc2_482_c7_input.out
u482c701r_flt.fits,WFALL

# ../pipe/cache/ingest/SVM/20231002/wfpc2_608_54/wfpc2_608_54_input.out
u6085401r_flt.fits,PC1
u6085402r_flt.fits,PC1
u6085403r_flt.fits,PC1
u6085404r_flt.fits,PC1
u6085405r_flt.fits,PC1
u6085406r_flt.fits,PC1
u6085407r_flt.fits,PC1
u6085408r_flt.fits,PC1
u6085409r_flt.fits,WF3
u608540ar_flt.fits,WF3

# ../pipe/cache/ingest/SVM/20231002/wfpc2_8lp_eg/wfpc2_8lp_eg_input.out
u8lpeg01m_flt.fits,WFALL-FIX

# ../pipe/cache/ingest/SVM/20231002/wfpc2_960_08/wfpc2_960_08_input.out
u9600801m_flt.fits,WFALL
u9600802m_flt.fits,WFALL
u9600803m_flt.fits,WFALL
u9600804m_flt.fits,WFALL

# ../pipe/cache/ingest/SVM/20231002/wfpc2_a01_02/wfpc2_a01_02_input.out
ua010201m_flt.fits,PC1-FIX
ua010202m_flt.fits,PC1-FIX
ua010203m_flt.fits,PC1-FIX
ua010204m_flt.fits,PC1-FIX
ua010205m_flt.fits,PC1-FIX
ua010206m_flt.fits,PC1-FIX

# ../pipe/cache/ingest/SVM/20231002/wfpc2_a01_11/wfpc2_a01_11_input.out
ua011101m_flt.fits,PC1-FIX
ua011102m_flt.fits,PC1-FIX
ua011103m_flt.fits,PC1-FIX
ua011104m_flt.fits,PC1-FIX
ua011105m_flt.fits,PC1-FIX
ua011106m_flt.fits,PC1-FIX
ua011107m_flt.fits,PC1-FIX

[stscijgbot-hstdp]

Comment by Steve Goldman on JIRA:

So, just to confirm what the desired outcome is, we want to use this new aperture column in the SVM poller file and have it added to the header keyword "APERTURE" in the output SVM drz file.

 

Is this keyword expected to have multiple values in only some cases (separated by semicolons)? 

 

Do we want any special logic for certain values? for example we have a number of extra rules for the filter keyword.

[stscijgbot-hstdp]

Comment by Lisa Sherbert on JIRA:

Yes, Steve Goldman, multi-values are separated by semi-colon: see HSTSDP-2166 for the list of the 77 distinct values that exist in the science table.   Seems like that works out well since the fields in the mis-named "poller" file content are separated by commas.

[stscijgbot-hstdp]

Comment by Michele De La Pena on JIRA:

Steve Goldman  Note the software which reads the ACS and WFC3 "full" (not just a list of filenames) poller files will not have the extra column at the end containing the aperture information.  This should not be an error.  Sorry, you probably know this, but it is on my mind.

Also, (1) users may only have a WFPC2 (or ACS, WFC3) file containing a list of filenames which will not contain any aperture information.  We need to discuss how this Use Case should be handled.

(2) While it seems that OPS will not produce WFPC2 MVM products, users may want to do this.  We need to consider that the random user should do regarding the APERTURE in the MVM case.

The APERTURE is only used for informational purposes (i.e., it does not drive any processing options) so this should be considered.

 

[stscijgbot-hstdp]

Comment by Steve Goldman on JIRA:

I've added a couple changes to a branch where I've passed the poller "aperture" to obs_info_dict['info'], which is stored as strings in the form: "11665 06 wfc3 ir ib4606cmq f160w drz". 

 

I'm now trying to find where this information is stored and passed to later in the code so that I can update the header keyword.

 

I still need to test the cases where their is no second column in the poller file, and make sure that an extra comma doesn't results in the overwriting of the APERTURE keyord with a nothing. 

[stscijgbot-hstdp]

Comment by Steve Goldman on JIRA:

I've found a way to pass the "aperture" keyword to the various FilterProduct, ExposureProduct, and TotalProduct objects, but the code is throwing an error that I will need to investigate. I've committed the changes to my branch (link above).

The way that the code currently parses the poller file using string indices is a nightmare and I would highly suggest this be refactored in the future.