NASA-PDS / operations

Tickets for the PDSEN Operations Team
Other
5 stars 1 forks source link

[nssdca-delivery] PDS_ATM Pack#6 #304

Closed tinagueth closed 1 year ago

tinagueth commented 1 year ago

Discipline Node Information

NOTE: If you have multiple delivery packages, we strongly encourage you to submit these in batches of 3 to 10 per issue with one ZIP file of the packages and another ZIP file of the validation reports. Please use a descriptive title, such as "Node Mission misc batch #".


Engineering Node Process

See the internal EN process at https://pds-engineering.jpl.nasa.gov/content/nssdca_interface_process

c-suh commented 1 year ago

@tinagueth, these 2 sets have been posted for NSSDCA processing. From tomorrow, you can check the status at https://nssdc.gsfc.nasa.gov/psi/ReportPDS4.jsp using the SIP LIDs below:

SIP LIDs: urn:nasa:pds:system_bundle:product_sip_deep_archive:insight_ps_v3.1_20221005 urn:nasa:pds:system_bundle:product_sip_deep_archive:insight_twins_v3.1_20221005

smclaughlin7 commented 1 year ago

@tinagueth @c-suh For this SIP LID

urn:nasa:pds:system_bundle:product_sip_deep_archive:insight_ps_v3.1_20221005

one collection product (urn:nasa:pds:insight_ps:data_raw::14.0) failed ingest because our ingest process detected a checksum mismatch for the collection label file at https://pds-atmospheres.nmsu.edu/PDS/data/PDS4/InSight/ps_bundle/data_raw/collection_ps_data_raw_inventory.xml

The SIP manifest file gives checksum fdd5f460c3f6813a75eabd926100dea7 but when ingest downloaded the label file, it computed checksum 25d5c9b796723827fbee8d2a6c360937.

This occurred because the Deep Archive Package was created on 2022-10-05 but the label file was changed on 2022-10-10. Our ingest process downloaded that file on 2022-10-12.

Please note we did successfully ingest and archive the basic products contained in this raw data collection; we just did not archive the collection product (label and .csv inventory file).

Solution: We recommend making a new Deep Archive Package for this bundle and submitting it to EN, but perhaps this is not necessary if you expect new version of the Insight PS bundle in the near future.

EN eventually plans to automate the checking and posting of Deep Archive Packages for the NSSDCA to pull (from https://pds.nasa.gov/data/pds4/manifests/) to minimize the time between when you deliver a package to EN and when the NSSDCA downloads and starts processing that package. Presently it usually takes several days depending on personnel availability, so it's best to wait, if possible, until the NSSDCA has fulling ingested a SIP before changing anything in bundle.

tinagueth commented 1 year ago

@c-suh @smclaughlin7 Thank you for the information. I recall doing the deep archive process but did not know about any changes made to the collection file afterwards. As far as I understand from Lyle, there will be one more InSight delivery some time next year (potentially January or March), which is, currently, also the final version. If that is too far along, especially if it is not until March, I can redo the deep archive part and resubmit it.

smclaughlin7 commented 1 year ago

@tinagueth @c-suh You're welcome, Tina. Since you expect the final version to be delivered next year, perhaps check with Lyle if the current version should be resubmitted. From the NSSDCA's standpoint, it's OK if you decide to wait and deliver the final version. If for some reason you need a copy of this current version, we can give you the entire bundle except for that raw data collection product that we could not ingest.

tinagueth commented 1 year ago

@smclaughlin7 @c-suh For consistency, and because we do not know when the next (or final) version will be done, it is best to make this ps_bundle up to date. I redid the deep archive process and submitted a new issue for it (see #308). I hope that resolved the collection product issue. Sorry about any inconvenience this might have caused. I appreciate the information.

smclaughlin7 commented 1 year ago

@tinagueth No worries! This did not cause any inconveniences at my end. It makes perfect sense to correct that collection product now and submit a new deep archive package for consistency and if the final version of the bundle may take longer than expected. Thanks Tina! @c-suh

c-suh commented 1 year ago

@tinagueth the set for insight_twins has been archived and the set for insight_ps has been resubmitted (through #308), so I will close this issue. Thank you for being so patient and for checking the status of many of your other sets after submission! The help is much appreciated!

tinagueth commented 1 year ago

@c-suh No worries. I was doing my part of our monthly report and was going through the submissions anyways, so I closed them if they were archived (you mentioned before that doing so was okay). Glad I can help out a little bit.