NASA-IMPACT / csdap-cumulus

SmallSat Cumulus Deployment
Other
1 stars 1 forks source link

Create a smoke test add on specifically for ORCA - For Metrics Milestone #271

Closed krisstanton closed 6 months ago

krisstanton commented 12 months ago

Ref: https://docs.google.com/spreadsheets/d/1Xznva0Upb9W9bTqdAndV4pDcwjV8jRRYKFVmgq_9i1c/edit#gid=0 Ref: 18.4.1 The system shall verify that data can be restored from ORCA Modify the ORCA smoke test to any specific specifications needed by the Data Metrics and Monitoring System. Related ticket: https://github.com/NASA-IMPACT/csdap-cumulus/issues/152

Most of this is covered within Ticket #152 The only additional work here would be to set up some kind of automated integrations into our Data Metrics system. Perhaps a mechanism for triggering an ORCA specific part of the smoke test for removing a test granule and then restoring it via ORCA, and then reporting back that was completed successfully.

krisstanton commented 9 months ago

Related Ticket (prerequisite / parent ticket) https://github.com/NASA-IMPACT/csdap-cumulus/issues/270

Update: The utility for comparing manifests verifies ORCA by proxy. The part of this ticket that is unique is to set this up to be part of a machine interface. The 'smoke test' part of this ticket is really not applicable since there is some what of a delay between an ingestion and an ORCA verification (it takes at least a few minutes to hours for an item to be replicated to ORCA, then it takes several hours up to a day before the manifest file is generated). So please keep in mind that any verification data is always about 1 day behind.

hbparache commented 6 months ago

Maybe don't need?

krisstanton commented 6 months ago

UPDATE: This is another ticket where recent discussions and plans have made this ticket no longer necessary. In detail: We are now going to have Manifests auto generated for all PROD (and ORCA) Files which are processed by the Observability Service DAG processes. This process will reconcile which granules are ingested, backed up and published.