Research and Prototype Earth Data CMR Requests for Granule Verification Logic - for DMS DAG
Background.
This a component is for the continuous deletion of files from the MCP Maxar Delivery Bucket.
This ticket encapsulates the task of checking if a Granules is properly published within the searchable Earth Data system.
[ ] Find out specifically which Earth Data CMR API Requests we can make in order to verify
[ ] Individual Granules
[ ] A Batch of Granules
[ ] Note, There is a normal API which we have to throttle requests, find out if there is an alternate one we can use, or at least what the limits are on the main one we can use.
[ ] Verify and document the CMR API Requests limits (for the normal and alternative method - if it exists)
[ ] Create a prototype which can make these requests and parse through the returned data
[ ] Use code I've previously written as a starting point
[ ] Ensure that each Granule we examine is actually stored on CBA PROD and NOT OLD NGAP (check the S3 Directory Path in the detailed metadata that comes back with the request)
[ ] Check with the DMS team to make sure this will work within the Archive DAG framework
[ ] Assist with integration of this code with the Archive DAG where needed.
Research and Prototype Earth Data CMR Requests for Granule Verification Logic - for DMS DAG
Background. This a component is for the continuous deletion of files from the MCP Maxar Delivery Bucket. This ticket encapsulates the task of checking if a Granules is properly published within the searchable Earth Data system.