Perform dry run of ingestion of WV04_Pan_L1B into CBA Prod

[chuckwondo]

At the time of this writing, CBA Prod should be nearly ready for operational usage, with the exception of configuring CloudFront to allow proper distribution via Cognito. Cognito itself is configured, but the routing of requests/redirects through CloudFront to the buckets is still pending some open support requests.

In addition, all thumbnails in the csdap-cumulus-prod-public bucket have been copied to the csdap-cumulus-prod-protected bucket, so that they are located with their related granule files, ready for discovery.

We want to test operation and metrics as best as possible until the final support requests are completed. To do so, we will perform a "dry run" of "migrating" the WV04_Pan_L1B collection from the current Prod account to the CBA Prod account. In preparation, we must do the following (in addition to copying of thumbnails, as mentioned above):

• [x] Update the bucket policy on csdap-cumulus-prod-protected to allow the CBA Prod account to "discover" the granules in the bucket. Specifically, we must add the following policy statement to the csdap-cumulus-prod-protected bucket (replacing the occurrences of ********1686 and ********5047 with the full account IDs of CBA UAT and Prod, respectively):

  {
      "Sid": "cumulus-allow-read-only-access",
      "Effect": "Allow",
      "Principal": {
            "AWS": [
                "arn:aws:iam::********1686:role/cumulus-uat_ecs_cluster_instance_role",
                "arn:aws:iam::********1686:role/cumulus-uat-lambda-processing",
                "arn:aws:iam::********5047:role/cumulus-prod_ecs_cluster_instance_role",
                "arn:aws:iam::********5047:role/cumulus-prod-lambda-processing"
            ]
        },
        "Action": [
            "s3:GetObject",
            "s3:GetObjectTagging",
            "s3:GetObjectVersionTagging",
            "s3:ListBucket"
        ],
        "Resource": [
            "arn:aws:s3:::csdap-cumulus-prod-protected/*",
            "arn:aws:s3:::csdap-cumulus-prod-protected"
        ],
        "Condition": {
            "StringEquals": {
                "aws:PrincipalOrgID": "o-j2j2wd84pp"
            }
       }
  }

• [x] Create new provider named "cumulus", which will refer to the csdap-cumulus-prod-protected bucket
• [x] Create a new rule (WV04_Pan_L1B_1) that uses the new "cumulus" provider, covers the entire temporal range of the collection, has a step of P1D, has a providerPathFormat of `"'WV04_PanL1B1/'yyyy/DDD", a hasmeta.discoverOnlyset totrue` (to indicate a "dry run" -- i.e., no queuing, and thus no posting to CMR)
• [x] Update the WV04_Pan_L1B___1_SmokeTest rule's provider and providerPathFormat to match that mentioned in the previous step
• [x] Restore the "QueueGranules?" step in DiscoverAndQueueGranules workflow to skip queuing when meta.discoverOnly is true
• [x] Add the following to the list of non-sandbox provider buckets in app/stacks/cumulus/tfvars/base.tfvars:

  cumulus = {
      name = "csdap-cumulus-prod-protected"
      type = "provider"
  }

• [x] Add logging to the formatProviderPaths, batchGranules, and unbatchGranules to aid in validating that they are operating as expected

Acceptance Criteria

Once the PR for this issue is approved, merged, and deployed to CBA UAT:

• [x] Add the new "cumulus" provider to CBA UAT
• [x] Updated WV04_Pan_L1B___1_SmokeTest rule in CBA UAT
• [x] Enable the updated WV04_Pan_L1B___1_SmokeTest rule in CBA UAT
• [x] Run the updated WV04_Pan_L1B___1_SmokeTest rule in CBA UAT
• [x] Confirm the rule runs successfully
• [x] Confirm that one of the ingested/updated cmr.json files has correctly updated the URLs that it contains

Once the above are confirmed in CBA UAT:

• [x] Approve the GitHub deployment to CBA Prod
• [x] Await successful deployment to CBA Prod
• [x] Add the new "cumulus" provider to CBA Prod
• [x] Add the WV04_Pan_L1B___1 collection to CBA Prod
• [x] Add the new WV04_Pan_L1B___1 rule to CBA Prod
• [x] Enable the new WV04_Pan_L1B___1 rule in CBA Prod
• [x] Run the new WV04_Pan_L1B___1 rule in CBA Prod
• [x] Confirm the rule runs successfully
• [x] Confirm the output from the lambda cumulus-prod-FormatProviderPaths indicates that it produced 2*365 = 730 dates/providerPaths
• [x] Confirm that there were 730 executions in the Map Run for DiscoverGranules
• [x] Confirm the output from the first of the UnbatchGranules executions (this will take a bit of work to determine how many items are expected to be in the batch)

[chuckwondo]

Running the WV04_Pan_L1B___1 rule in CBA Prod failed. I opened a new issue for this: https://github.com/NASA-IMPACT/csdap-cumulus/issues/296

Once the new issue is fixed, I'll rerun the rule, and continue with the subsequent steps above.

[chuckwondo]

After fixing #296, another error occurred, as reported in #300

[chuckwondo]

Successfully ran dry run (discovery only; no ingest/publish) of WV01_Pan_L1B in its entirety, which spans 2007-2022, for a total of 5844 days. It ran in 1:24:34.