Collect CNM content and write it into CNM template

[lisakaser]

3 story points

Acceptance criteria:

Scenario

GIVEN: App installed and example ini & UMM-G available When: Run the app with TBD command line option THEN: Will generate CNM file Visually inspect output for correctness/completeness

[juliacollins]

@afitzgerrell here's some sample output. The contents of the init file used in this experiment are:

[Source]
data_dir: ./data/modscgdrf

[Collection]
auth_id: NSIDC-FOOBAR
version: 1
provider: FORTYTWO

[Destination]
local_output_dir: ./json
ummg_dir = ummg
kinesis_arn: abcd-1234-wxyz-0666

From the command line: instameta process --config example/template.ini --env=hoo I used hoo as the environment so you could see where it ends up in the s3 urls. :-)

Resulting CNM output is in the attached files (you may need to reformat/use a pretty printer):

h08v05_Terra_20220415.v2023.1.nc.cnm.json h07v03_Terra_20220415.v2023.1.nc.cnm.json

[juliacollins]

@afitzgerrell Kevin still needs to review my code, but you can review the functionality now if you like. See issue-39 for steps to update your working directory. I recommend you create a new config file and then try out the process command with that file. You'll need to be running a poetry shell as well. The steps are:

• poetry shell (start up the shell)
• poetry install (make sure all dependencies are installed)
• instameta init (create a new config file. You can specify a file to create with --config my-config-file-name or you will be prompted to enter a file name if needed)
• instameta process --config config-file-from-previous-step --env=myenv (process the files that should be in data/modscgdrf using the environment myenv in the s3 url)

Note that -- before the command line options is two dashes. The output should be self-explanatory, but let me know if it's not!

[afitzgerrell]

Complete. I created a fitzTest.ini and generated .cnm.json message files (which I've since cleaned out of the directories).

But, for posterity, this is the content from one of the .cnm.json files output:

{ "version": "1.5.1", "submissionTime": "2024-09-05T19:51:35.873904+00:00", "identifier": "0e957e3e-a5ae-4b37-8d1c-01a9407d59f7", "collection": "nsdic-1234", "provider": "DPT", "product": { "name": "h08v05_Terra_20220415.v2023.1.nc", "dataVersion": "999", "files": [{ "name": "h08v05_Terra_20220415.v2023.1.nc", "type": "data", "uri": "s3://nsidc-cumulus-hoo-ingest-staging/external/nsdic-1234/999/0e957e3e-a5ae-4b37-8d1c-01a9407d59f7/h08v05_Terra_20220415.v2023.1.nc", "checksumType": "SHA256", "checksum": "82ea4a8783e1bc083a14ae9916b78a66c2eb5af0b990b4f5053dbd499a233905", "size": 3472582 }, { "name": "h08v05_Terra_20220415.v2023.1.nc.json", "type": "metadata", "uri": "s3://nsidc-cumulus-hoo-ingest-staging/external/nsdic-1234/999/0e957e3e-a5ae-4b37-8d1c-01a9407d59f7/h08v05_Terra_20220415.v2023.1.nc.json", "checksumType": "SHA256", "checksum": "a0c16de0e99bc2d508d0be2cbf1671e106a65546788fe0b1605dee16a9e9f513", "size": 1614} ] } }

and it contains everything I would expect/hope for based on sample CNM message content currently shown in the CNM guidelines as of 5 Sept 2024.