nasa / opera-sds-pcm

Observational Products for End-Users from Remote Sensing Analysis (OPERA)
Apache License 2.0
16 stars 12 forks source link

[New Feature]: DSWx-S1 Validator Script for OPS Production Audits #888

Closed riverma closed 2 months ago

riverma commented 4 months ago

Checked for duplicates

Yes - I've already checked

Alternatives considered

Yes - and alternatives don't suffice

Related problems

The current DSWx-S1 validator script returns only a list of MGRS tile set IDs. For OPS to use this information to audit whether processing is performing correctly and delivering the right products to CMR, this information needs to be manually converted to expected granule IDs that CMR should contain and OPS needs to manually check CMR.

Describe the feature request

The validator script should include an additional feature that takes the list of expected tile set IDs, and returns the list of granule IDs expected to be produced while also checking CMR if they are already delivered.

riverma commented 4 months ago

@LalaP: what information does OPS need to verify DSWx-S1 processing? Please let me know if what I captured above is sufficient. @hhlee445: we need delivery information of DSWx-S1 at CMR, like collection name, granule ID signature. Could you help provide:

riverma commented 4 months ago

Update on the second bullet point above:

Looked into it and i don't think there's a direct predictive quality to the DSWx-S1 granule ID signature that will tie back to input RTCs or MGRS Tile Set IDs. I think we'd have to search the DSWx-S1 collection and request matches for the InputGranules field against the input RTCs for a given product we think should exist.

See the Metadata tab for this example granule: https://search.uat.earthdata.nasa.gov/search/granules/granule-details?p=C1261113948-POCLOUD&pg[0][v]=f&pg[0][gsk]=-start_date&g=G1264681475-POCLOUD&q=DSWX&tl=1719277753!3!!

hhlee445 commented 4 months ago

mgrs_set_id maps to mgrs_tiles

example:

MS_112_60 -> mgrs_tiles list ['23VNJ', '23VNK', '23VPJ', '23VPK', '24VUP', '24VUQ', '24VVP', '24VVQ']

  1. OPERA_L3_DSWx-S1_T23VNJ_20240622T091336Z_20240625T030625Z_S1A_30_v0.4
  2. OPERA_L3_DSWx-S1_T23VNK_20240622T091320Z_20240625T030631Z_S1A_30_v0.4
  3. OPERA_L3_DSWx-S1_T23VPJ_20240622T091333Z_20240625T030638Z_S1A_30_v0.4
  4. OPERA_L3_DSWx-S1_T23VPK_20240622T091318Z_20240625T030549Z_S1A_30_v0.4
  5. OPERA_L3_DSWx-S1_T24VUP_20240622T091332Z_20240625T030557Z_S1A_30_v0.4
  6. OPERA_L3_DSWx-S1_T24VUQ_20240622T091318Z_20240625T030604Z_S1A_30_v0.4
  7. OPERA_L3_DSWx-S1_T24VVP_20240622T091329Z_20240625T030611Z_S1A_30_v0.4
  8. OPERA_L3_DSWx-S1_T24VVQ_20240622T091316Z_20240625T030618Z_S1A_30_v0.4

MS_112_59 -> mgrs_tiles list ['23VNL', '23VPL', '23WNM', '23WPM', '24VUR', '24VVR', '24VWR', '24WVS', '24WWS']

  1. OPERA_L3_DSWx-S1_T23VNL_20240622T091306Z_20240625T025330Z_S1A_30_v0.4
  2. OPERA_L3_DSWx-S1_T23VPL_20240622T091305Z_20240625T025402Z_S1A_30_v0.4
  3. OPERA_L3_DSWx-S1_T23WNM_20240622T091258Z_20240625T025341Z_S1A_30_v0.4
  4. OPERA_L3_DSWx-S1_T23WPM_20240622T091258Z_20240625T025320Z_S1A_30_v0.4
  5. OPERA_L3_DSWx-S1_T24VUR_20240622T091303Z_20240625T025413Z_S1A_30_v0.4
  6. OPERA_L3_DSWx-S1_T24VVR_20240622T091302Z_20240625T025351Z_S1A_30_v0.4
  7. OPERA_L3_DSWx-S1_T24VWR_20240622T091259Z_20240625T025444Z_S1A_30_v0.4
  8. OPERA_L3_DSWx-S1_T24WVS_20240622T091259Z_20240625T025425Z_S1A_30_v0.4
  9. OPERA_L3_DSWx-S1_T24WWS_20240622T091259Z_20240625T025435Z_S1A_30_v0.4
riverma commented 4 months ago

Thank you @hhlee445!