[nssdca-delivery] urn:nasa:pds:maven.rose.calibrated::1.18

bharris-igpp commented 1 year ago

Discipline Node Information

Delivering Node: please enter your DN here for record keeping purposes PPI
NSSDCA Delivery Package: please upload the files output by PDS Deep Archive as a TAR.GZ or ZIP file here, or supply a URL to download from

maven-rose-cal_v1.0_2022-08-24.zip

Validation report: please upload a TXT report or screenshot of PDS4 Validate Tool run on your bundle. NSSDCA only accepts valid PDS4 bundles. be sure you run validate with the -R pds4.bundle flag enabled to ensure all integrity checks are completed successfully.

maven-rose-cal_VAL_2022-08-24.txt

NOTE: If you have multiple delivery packages, we strongly encourage you to submit these in batches of 3 to 10 per issue with one ZIP file of the packages and another ZIP file of the validation reports. Please use a descriptive title, such as "Node Mission misc batch #".

Engineering Node Process

See the internal EN process at https://pds-engineering.jpl.nasa.gov/content/nssdca_interface_process

c-suh commented 1 year ago

@bharris-igpp, this set has been posted for NSSDCA processing. From tomorrow, you can check the status at https://nssdc.gsfc.nasa.gov/psi/ReportPDS4.jsp using the SIP LID below:

SIP LID:

urn:nasa:pds:system_bundle:product_sip_deep_archive:maven.rose.calibrated_v1.18_20220825

smclaughlin7 commented 1 year ago

@jordanpadams @c-suh @bharris-igpp This SIP LID:

urn:nasa:pds:system_bundle:product_sip_deep_archive:maven.rose.calibrated_v1.18_20220825

hit our NSSDCA ingest queue this morning, so we will ingest and archive it.

Issue

This delivery only contained 2 products: bundle and document collection. The PDS Deep Archive tool did not include the other primary collections, data and calibration, in the delivery SIP manifest because the VIDs specified in the bundle product:

urn:nasa:pds:maven.rose.calibrated:data.sky::**_1.15_** urn:nasa:pds:maven.rose.calibrated:calibration.fup::**_1.15_**

did not match the VIDs specified in the collection products' labels:

urn:nasa:pds:maven.rose.calibrated:data.sky **_1.17_** urn:nasa:pds:maven.rose.calibrated:calibration.fup **_1.17_**

The Deep Archive tool took he expected action for this case, where it could not find collections associated with the lidvid_references specified in the bundle product.

Interestingly, the PDS4 Validation tool only flagged these the mismatches as referential integrity warnings. Here's the abbreviated Validation report that Bill provided, which shows the warnings: maven-rose-cal_VAL_2022-08-24_abbreviated.txt

Proposal for PDS-Ops (Jordan, Catherine)

When a Validation report identifies one or more referential integrity warnings, could PDS-Ops ask the submitting Node to investigate these warnings before EN-Ops posts the Deep Archive package (SIP) for the NSSDCA to pull?

Does/should the Deep Archive tool throw a warning if the bundle product specifies a primary collection by but it cannot find a collection product with that LIDVID.

Proposed Action for PPI Node (Bill)

To archive the entire bundle, the NSSDCA recommends correcting the VIDs in the bundle product, then generate a new Deep Archive package and deliver it to PDS-Ops.

Bill, Could you please wait until we have ingested this SIP to edit the bundle product? Any time after Sept. 14 should be OK. Thanks for your patience.

Thanks all!

smclaughlin7 commented 1 year ago

@bharris-igpp We/NSSDCA have downloaded and ingested this submission and it's in the queue to be written to tape

urn:nasa:pds:system_bundle:product_sip_deep_archive:maven.rose.calibrated_v1.18_20220825

It's now safe to edit the bundle product. (No need to wait until after Sept. 14.) Thanks!

cc: @c-suh @jordanpadams

jordanpadams commented 1 year ago

@smclaughlin7

When a Validation report identifies one or more referential integrity warnings, could PDS-Ops ask the submitting Node to investigate these warnings before EN-Ops posts the Deep Archive package (SIP) for the NSSDCA to pull? We can add to our procedure to check for this warnings and request the nodes review them. @c-suh

Does/should the Deep Archive tool throw a warning if the bundle product specifies a primary collection by but it cannot find a collection product with that LIDVID. Not sure. But this should be caught and reported by validate anyways.

smclaughlin7 commented 1 year ago

@jordanpadams Thanks! Adding a check to your procedure to look for referential integrity warnings and request the nodes review will help. @c-suh

Right -- mismatches between the bundle product and a collection product would be caught and validated via referential integrity. Given the above extra check, the Deep Archive shouldn't need to throw a warning.

c-suh commented 1 year ago

Hi @smclaughlin7 I was checking the status of this package, maven.rose.calibrated_v1.18_20220825, and see that it's been "Processed with Exception" due to the two files you mention above. Just to clarify, are we waiting on @bharris-igpp to fix the mismatch of versions in the bundle product and resubmit the package (in this ticket)?

smclaughlin7 commented 1 year ago

Hi @smclaughlin7 I was checking the status of this package, maven.rose.calibrated_v1.18_20220825, and see that it's been "Processed with Exception" due to the two files you mention above. Just to clarify, are we waiting on @bharris-igpp to fix the mismatch of versions in the bundle product and resubmit the package (in this ticket)?

Hi @c-suh! That's correct. We are waiting on @bharris-igpp to fix the mismatch of versions in the bundle product and resubmit the package (in this ticket). Thanks for checking about this issue.

c-suh commented 9 months ago

@bharris-igpp we are closing this ticket. Please open a new one with the corrected package when it is ready. Thank you!

NASA-PDS / operations