Ingest WV03_Pan_L1B granules for year 2017 into Production

[jsrikish]

• [x] Ask Brad or Madhu to restore WV03_Pan_L1B granules for 2017 from Glacier (15 days) and generate UMM-G metadata files
• [x] Once all metadata files are generated, in Prod, run Cumulus rule named WV03_Pan_L1B___1_2017:

  DOTENV=.env.prod make bash
  cumulus rules enable --name WV03_Pan_L1B___1_2017
  cumulus rules run --name WV03_Pan_L1B___1_2017

• [x] Query the CMR to check the hits count, indicating the granule count: https://cmr.earthdata.nasa.gov/search/granules.umm_json?short_name=WV03_Pan_L1B&temporal=2017-01-01T00:00:00Z,2017-12-31T23:59:59.999Z
• [ ] Confirm that ingested granules are visible via Earthdata Search: https://search.earthdata.nasa.gov/search?q=WV03&qt=2017-01-01T00%3A00%3A00.000Z%2C2017-12-31T23%3A59%3A59.999Z
• [ ] Find a resolution to the missing granules - linked ticket - https://github.com/NASA-IMPACT/csdap-cumulus/issues/308

[jsrikish]

In progress

[jsrikish]

Earthdata: 357,165 granules Airflow: 420,634 granules Difference: 64,469. ???

[jsrikish]

Need to go into the dashboard to look at the error messages for the granules discrepancy

[jsrikish]

Errors:

1. "errorType": "Error", "errorMessage": "At least 1 granule in the input list is missing a file object representing a CMR metadata file. This indicates that either each such granule indeed has no such file amongst all of its files within the provider's bucket (in which case, you must populate them), or that the collection is misconfigured such that the CMR files are not identified during discovery (either because the 'granuleIdExtraction' regular expression is incorrect, or 'ignoreFilesConfigForDiscovery' is set to 'false' [or not specified] and the 'files' list does not include it or it is misconfigured):
   2. 1 file had file length mismatch

[chuckwondo]

@jsrikish, where does this stand? Is this something that needs to be resolved before we switch to CBA Prod?

[hbparache]

@jsrikish will do a count of the "launchpad" error, and divide by 5 or 6 to determine how many of the granule failures were due to this error

[hbparache]

Jayanthi has a number - needs to run by Kris and Chuck

[hbparache]

Kris will take a look at the log file in the next couple of weeks (received from Jayanthi last week)

[krisstanton]

I made a custom version of the log aggregator just to look at the large log file Jayanthi had provided me with. Almost every single one of them are launch pad errors, which I believe may be resolved now. If we get more of these errors on the latest round of migration ingests and normal ingests, we should revisit this.