search

NASA-PDS / operations

Tickets for the PDSEN Operations Team
Other
5 stars 1 forks source link

[nssdca-delivery] urn:nasa:pds:insight_documents::2.0 and urn:nasa:pds:insight_hp3_tem::1.0 and urn:nasa:pds:insight_rad::2.1 and urn:nasa:pds:insight_rise_derived::1.0 and urn:nasa:pds:insight_rise_raw::2.0 and urn:nasa:pds:insight_seis::3.0 #511

Open gbowen99 opened 1 month ago

gbowen99 commented 1 month ago

Discipline Node Information

gbowen99 commented 1 month ago

The Geosceinces node is aware of the errors present in the validation reports for urn:nasa:pds:insight_documents::2.0 and urn:nasa:pds:insight:seis::3.0. We are not planning on correcting them at this time.

smclaughlin7 commented 1 month ago

@c-suh @jordanpadams Could you please hold off checking and posting this SIP LID?

I see extraneous data files in the SIP manifest. I highlighted those files in the attached XLS version of the manifest: insight_documents_v2.0_20240429_sip_v1.0.tab.xlsx. The extraneous files are for collections having secondary memebership in the bundle and basic products in the secondary collections.

The only files that should be listed in a PDS4 SIP manifest are:

  1. files of the bundle product (e.g., bundle.xml, readme.txt)
  2. files of collection products (e.g., collection.xml and its inventory) having primary membership in the bundle
  3. files of basic products having primary membership in those collections

I have not see this for SIP manifests for ATM's Mars2020 MEDA bundle (https://pds-atmospheres.nmsu.edu/PDS/data/PDS4/Mars2020/mars2020_meda/bundle.xml). That bundle identifies several collections with secondary membership. To date, ATM has submitted it eight times and the manifests never included secondary collection products and their basic products.

Could this have something to do with how a node generates the Deep Archive package, e.g., via local file system or via registry? Should I open a new ticket for Deep Archive for this case?

Thanks.

jordanpadams commented 1 month ago

@smclaughlin7 yes. this is because they list them as secondary but have the files on the file system. we will have to file a ticket for this.

@nutjob4life can you create a new ticket for this and investigate? I would work with @jshughes to understand more details about primary vs. secondary products/collections.

jordanpadams commented 1 month ago

@nutjob4life this may also be something we didn't think of with pds-deep-registry-archive and accessing that information from the API. I don't think the API cares about primary vs. secondary, so it may cause some issues...

nutjob4life commented 1 month ago

Hi @jordanpadams, my access to my laptop is restored (thanks JPL help desk—took about an hour) and I will set up the ticket.

Reminder: I'm on vacation today through May 24th.

c-suh commented 1 month ago

@gbowen99 thank you for the note about the errors for documents 2.0 and seis 3.0. As @smclaughlin7 has requested, I have not posted the package for documents 2.0 but the other 4 sets have been posted for NSSDCA processing. From tomorrow, you can check the status at https://nssdc.gsfc.nasa.gov/psi/ReportPDS4.jsp using the SIP LIDs below:

SIP LIDs:

For the last package for rise, there are a number of validate errors (see attached report). Please let me know if these are errors you will be addressing or ignoring, such as done with documents 2.0 and seis 3.0. Thank you.

gbowen99 commented 3 weeks ago

@c-suh sorry for the late response!

I am a bit confused by the report you attached for the rise_raw dataset. Is that a validation report that you put together? The report I submitted for the bundle used validate 3.5.0-SNAPSHOT and doesn't have any errors present. Any idea what might be going on?

c-suh commented 4 days ago

Hi @gbowen99 - my apologies for the delay. Yes, the report that I attached above is one that I created. We ask the nodes to run the Validate tool with certain parameters, and then we at the Engineering node run the Validate tool again with different parameters. Since a new version of Validate was released last week, I did this again and see the same errors. I will check with our developers to make sure this isn't a bug on our part and will get back to you. Thank you for your patience!

c-suh commented 4 days ago

@gbowen99 this is an issue with our previous deep archive tool. If you have not already, please install the latest version, which has the bug fix for this error, to use that moving forward. For the problematic insight_rise_raw package, we will address the error then post this for NSSDCA processing; there is nothing more for you to do regarding this ticket for the time being (I expect documents 2.0 will go on another ticket after we implement the necessary functionality to the deep archive tool). Thank you!