Wrong input file dimensions in calwebb_coron3

[stscijgbot]

Issue JP-831 was created by Howard Bushouse:

JWSTDMS-33 reports an error running the calwebb_coron3 pipeline on a MIRI observation, in which the datamodels schema validation fails when loading the PSF exposures from the input ASN, saying that the data are only 2-D when it's expecting 3-D (trying to load into a CubeModel).

Inputs to coronagraphic processing should be multi-integration data, i.e. rateints, calints, and hence should be 3-D.

[stscijgbot]

Comment by Howard Bushouse: Example data files for one of the problem cases have been copied to /grp/jwst/ssb/chartreuse/dms-bugs/jwstdms-33/. The coron3 ASN file shows 3 instances of a "science" or "psf" member being listed as a "cal" product, when it should be using "calints" as input: {code:java}                 {                     "expname": "jw00628020001_04101_00001_mirimage_cal.fits",                     "exptype": "science",                     "exposerr": "null",                     "asn_candidate": "('c1014', 'coronagraphy')"                 },                 {                     "expname": "jw00628017001_04101_00001_mirimage_cal.fits",                     "exptype": "psf",                     "exposerr": "null",                     "asn_candidate": "('c1014', 'coronagraphy')"                 },                 {                     "expname": "jw00628014001_04101_00001_mirimage_cal.fits",                     "exptype": "science",                     "exposerr": "null",                     "asn_candidate": "('c1014', 'coronagraphy')"                 }, {code} There are also several "target_acquisition" members that use "cal" products as input, but that's correct for TA's and all of those members should be skipped in calwebb_coron3 processing anyway. The pool file from which this ASN was created is available in the test data directory.

So this appears to be a problem related to ASN construction. The 3 "cal" products listed as either "science" or "psf" exptype should be "calints" or they shouldn't be in the ASN at all.

[stscijgbot]

Comment by Howard Bushouse: Further investigation has shown that the ASN pool entries for these exposures have EXP_TYPE='MIR_LYOT', which is incorrect, because they are in fact TA exposures and hence should have EXP_TYPE='MIR_TACQ'. The bad EXP_TYPE values in the pool in turn causes them to get listed as "science" members in the ASN file, when in fact they should be labeled as "target_acquisition".

Mike Swam indicates that the bad pool entries are due to mismatches between the PSSDB and OSF for this proposal: {noformat} This is a case where the PPS and OSF have different content, and we need to form the new observation_identifier rows in PPS to link them up.

The 3rd tacq was not taken, so the current code is mis-aligning the OSF science row1 with PPS tacq row3.{noformat} SDP will fix this issue, which will in turn give proper values in the ASN pool.

[stscijgbot]

Comment by Howard Bushouse: The only remaining issue that should be checked in Cal is whether the "calwebb_coron3" pipeline is setup to only load/process ASN members with exptypes of "science" and "psf", thus causing anything like target-acqs to be skipped.

[stscijgbot]

Comment by David Davis: Once the pool files are fixed we should retest but the code currently selects only files with exp_type='science' and 'psf' {code:java}

Construct lists of all the PSF and science target members

psf_files = [m['expname'] for m in prod['members'] if m['exptype'].upper() == 'PSF'] targ_files = [m['expname'] for m in prod['members'] if m['exptype'].upper() == 'SCIENCE']{code}

[stscijgbot]

Comment by Howard Bushouse: JWSTDMS-33 has been reassigned to SDP for fix the PPSDB/OSF alignment issues.