search

NASA-PDS / operations

Tickets for the PDSEN Operations Team
Other
5 stars 1 forks source link

[nssdca-delivery] urn:nasa:pds:juno_jiram #345

Closed tinagueth closed 1 year ago

tinagueth commented 1 year ago

Discipline Node Information

NOTE: If you have multiple delivery packages, we strongly encourage you to submit these in batches of 3 to 10 per issue with one ZIP file of the packages and another ZIP file of the validation reports. Please use a descriptive title, such as "Node Mission misc batch #".

Engineering Node Process

See the internal EN process at https://pds-engineering.jpl.nasa.gov/content/nssdca_interface_process

smclaughlin7 commented 1 year ago

@c-suh Catherine, a heads up -- This delivery package uses the "2022" subdirectory in the in the juno_jiram_v1.1_20221215_sip_v1.0.xml label file. Please post this package in "2022" subdir: https://pds.nasa.gov/data/pds4/manifests/2022/. Is the year string an input run parameter for the Deep Archive Tool, or perhaps it is hardcoded in the software (seems unlikely)? I'd like to understand going forward if the nodes need to adjust an input parameter or if the software needs to be updated for the new year. Thanks!

@tinagueth You do not have to do anything about this delivery package.

c-suh commented 1 year ago

@smclaughlin7, regarding the subdirectory to which the package is posted, our script will (now) automatically post it to the year's subdirectory specified in the . (I implemented this after our exchange at the beginning of this year but very much appreciate the heads up and feel slightly gleeful that you won't need to pay attention to this (for us) moving forward!)

Regarding the question for the Deep Archive Tool, that is a good question and one for @jordanpadams:

Is the year string an input run parameter for the Deep Archive Tool, or perhaps it is hardcoded in the software (seems unlikely)? I'd like to understand going forward if the nodes need to adjust an input parameter or if the software needs to be updated for the new year. Thanks!

smclaughlin7 commented 1 year ago

@c-suh Excellent! Your script will now auto-post to the year subdir specified in . The trick, or perhaps the challenge, for me will be to remember that in ~11 months! ;-)

jordanpadams commented 1 year ago

@smclaughlin7 the year in the diretory is added based upon the datetime when deep archive was executed.

c-suh commented 1 year ago

@smclaughlin7 hopefully it won't be an issue in ~11 months and thus won't require you to remember anything! :]

@tinagueth this set has been posted for NSSDCA processing. From tomorrow, you can check the status at https://nssdc.gsfc.nasa.gov/psi/ReportPDS4.jsp using the SIP LID below:

SIP LID:

smclaughlin7 commented 1 year ago

@tinagueth Hi Tina, Our ingest process failed these two collection products

urn:nasa:pds:juno_jiram:data_raw::1.0
urn:nasa:pds:juno_jiram:data_calibrated::1.0

because the checksums for the inventory files that we downloaded last week did not match the checksums listed in the SIP manifest file https://pds.nasa.gov/data/pds4/manifests/2022/juno_jiram_v1.1_20221215_sip_v1.0.tab. The timestamps on the two inventories indicate the files were changed in Jan. 2023, after the Deep Archive/SIP package was generated in mid-Dec. (We did successully ingested the primary basic products associated with these collections when the SIP was generated, so no worries there.) In the future would it be possible wait until the NSSDCA have ingested a SIP before changing a collection inventory?

Also we've received these two collections four times already, but their product VIDs have always remained at 1.0. We recommend incrementing a collection's VID whenever its inventory file or its XML label is changed. MIght that be acceptable for this bundle?

Thanks!

cc: @c-suh

tinagueth commented 1 year ago

@smclaughlin7 @c-suh not sure if mis-match is due to Lyle having to redo the validation report as several issues were present after his initial run and me not re-running the deep archive part after he fixed those issues. If that could be the reason, I will generate the deep archive again for such occurrences in the future. I forwarded your collection VID comment to Lyle and will check with him about that for future submissions. Thanks

smclaughlin7 commented 1 year ago

@tinagueth @c-suh Thank you checking tinto these two items. Much appreciated!

tinagueth commented 1 year ago

@smclaughlin7 @c-suh Not a problem. I forgot to ask if I should redo the deep archive run and re-submit it to fix the two collection products (no the VID part currently) or if it is okay. Thanks

smclaughlin7 commented 1 year ago

@tinagueth It's up to you if you/ATM want to redo and resubmit a new deep archive package so that we can ingest the Jan 2023 instance of those two collection products. If you expect more data to soon be appended to the Juno JIRAM bundle and a new deep archive package to be made shortly thereafter, you may just want to wait. Thanks for asking!

tinagueth commented 1 year ago

@smclaughlin7 as the next Juno release is coming next month, I will leave this submission as is (due to holidays and such, the last Juno jiram took longer to get fully submit then expected, though that should not be the case for the next release). Also, Lyle replied in an email in regard to the collection VID comment, so I will not copy his reply here. Thanks

smclaughlin7 commented 1 year ago

@tinagueth This plan makes sense to me. Thanks!

smclaughlin7 commented 1 year ago

@tinagueth Good news. The NSSDCA finished archiving this SIP LID to tape, so we're done:

I just want to reiterate my comment from Feb 16 that we did not ingest these two collection products

urn:nasa:pds:juno_jiram:data_raw::1.0
urn:nasa:pds:juno_jiram:data_calibrated::1.0

because the checksums listed in the SIP manifest file did not much the checksums for the downloaded files but we did successfully ingest and archive all basic products associated with those two collection products.

tinagueth commented 1 year ago

@smclaughlin7 Thank you for the update. I did inform Lyle about this back then. Not sure if that issue has been resolved with the newest JUNO jiram release.

tinagueth commented 1 year ago

@c-suh SIP/AIP for this bundle has been archived.