Daily Summary Report

[viviant100]

Update (1/3/24): Leverage EDC Metrics for reporting and OPS monitoring

• Prerequisite: External Cloud Metrics Team configures SWODLR log ingest in Cloud Metrics UAT/Prod venue.
• Once the Cloud Metric Team import the CloudWatch logs to ElasticSearch, our team (SWODLR & OPS) can set up the dashboard and tools tailored to our needs.
• Previous suggested dynamo DB dump approach requires us to read the entire database, filter the timestamp, etc. every night, which can be improved by leveraging EDC metrics.

9/27/23: This daily report provides an overview of granules:

• ingested (PIXC/PIXCVec/Raster)
• created (L2_HR_Raster) - the initial request from the user was submitted
• published (L2_HR_Raster) - the product was uploaded to S3 and the user is able to download
• downloaded (L2_HR_Raster)

Options: (1) Use RDS to generate a daily summary report (work with Jackie to get a operation report e.g. db dump) (2) Develop a dedicated SWODLR dashboard to monitor Cloud Metrics.

Refer to

• OPERA visual report for ingestion, creation and publication. (based on Cumulus ingestion)
• RDS for backfilling tool
  • Nightly report with a query for
    • each collection
    • query result is exported to a table and put on lapinta
    • status of user requests (e.g completion and failures)

[viviant100]

closed due to duplication of swodlr#108