NASA-PDS / operations

Tickets for the PDSEN Operations Team
Other
5 stars 1 forks source link

[nssdca-delivery] PDS_ATM Pack #9 #373

Closed tinagueth closed 9 months ago

tinagueth commented 1 year ago

Discipline Node Information

NOTE: If you have multiple delivery packages, we strongly encourage you to submit these in batches of 3 to 10 per issue with one ZIP file of the packages and another ZIP file of the validation reports. Please use a descriptive title, such as "Node Mission misc batch #".


Engineering Node Process

See the internal EN process at https://pds-engineering.jpl.nasa.gov/content/nssdca_interface_process

c-suh commented 1 year ago

Discovered a bug which must be addressed first. Ticket for the bug is here.

jordanpadams commented 1 year ago

@tinagueth per some discussion offline, it appears your file_name attribute is using a relative path. it appears there may be a bug in the PDS4 Information Model specification, which is why Validate is not catching it. you should use directory_path_name to specify the path: https://pds.nasa.gov/datastandards/documents/im/v1/index_1J00.html#class_pds_document_file

tinagueth commented 1 year ago

@jordanpadams Thank you for the information. I did not know about this. How to I specify the path for this keyword though? Is it where it points to on the website (i.e. https://pds-atmospheres.nmsu.edu/PDS/data/vo_3002/catalog/)? I tried that but get a red flag in oxygen, with the "quick fix solutions" being to add file_name that I just removed. I added it back (still red flag) until I removed the logical_identifier part, which then gave a green flag. So currently I have:

dataset.cat https://pds-atmospheres.nmsu.edu/PDS/data/vo_3002/catalog/ 7-Bit ASCII Text

note that I also tried removing the file_name part and adding dataset.cat to the end of the directory_path_name

https://pds-atmospheres.nmsu.edu/PDS/data/vo_3002/catalog/dataset.cat 7-Bit ASCII Text

but get the same red flag in oxygen with quick solution being adding File_name. Am I missing something? Creating an xml file for a catalog file is new to me (I have done document files before), but something that I will need to do for other datasets. Thank you

jordanpadams commented 1 year ago

@tinagueth I would work with some other folks on your team, but you need both file_name and directory_path_name. The path should be the relative path from the label file. That being said, using ../ seems a little funky. I don't see it is explicitly disallowed by PDS4, but it seems like it could get a little out of hand. I am having some folks raise this up to the DDWG. Usually directory_path_name is for directories further down in the directory tree.

e.g. dataset.xml -- path/ ---- to/ ------ data/ -------- dataset.cat

<file_name>dataset.cat</file_name>
<directory_path_name>path/to/data</directory_path_name>
smclaughlin7 commented 1 year ago

@tinagueth I have not yet used a relative path in a PDS4 product, so I decided to learn about it by playing with https://pds-atmospheres.nmsu.edu/PDS/data/vo_3002/document/dataset.xml in Oxygen.

I was surprised when Oxygen validated both of these two _incantations for _:

[1]
<Document_File>
   <file_name>dataset.cat</file_name>
   <local_identifier>vo_irtm_dataset_cat</local_identifier>
   <directory_path_name>../catalog/</directory_path_name>
   <document_standard_id>7-Bit ASCII Text</document_standard_id></Document_File>

[2]
<Document_File>
   <file_name>dataset.cat</file_name>
   <local_identifier>vo_irtm_dataset_cat</local_identifier>
   <directory_path_name>https://pds-atmospheres.nmsu.edu/PDS/data/vo_3002/catalog/</directory_path_name>
   <document_standard_id>7-Bit ASCII Text</document_standard_id
</Document_File>

I expected only [1] to validate because Standards Ref Section 2B.1.1 Objects and Files states "A label and all of its associated digital objects must be stored in the same directory unless the directory_path_name attribute is available, in which case directory_path_name may be used to indicate a subdirectory relative to the directory containing the label." Incantation [2] is not a relative path so I expected Oxygen to flag it. (I did not run [1] and [2] through the PDS Validate tool.)

Hmm...Perhaps this relates to the possible IM bug that @jordanpadams mentioned earlier.

jordanpadams commented 1 year ago

@smclaughlin7 the latter [2] option I think is a bug in the IM where directory_path_name is a string, it is not ASCII_Directory_Path, which does more refined checked.

tinagueth commented 1 year ago

@jordanpadams @smclaughlin7 @c-suh Thanks for the updates. For now, I will hold off on this until I had a chance to talk to Lyle about it, and see what might come out of the DDWG meeting. Is the reference by @smclaughlin7 from a handbook that I should know about? I checked my notes from Joni about PDS4 NSSDCA but she did not provide a link to a handbook. Thank you.

smclaughlin7 commented 1 year ago

@tinagueth Oops. I should've linked to the PDS4 Standards Reference document. It's one of the two fundamental PDS4 documents; the other is the PDS Common Data Dictionary. Both are available at https://pds.nasa.gov/datastandards/documents/. I recommend bookmarking that webpage. You can also reach the PDS4 Documents via the PDS mainpage: https://pds.nasa.gov --> click the "DATA STANDARDS" tab in the gray top-nav bar --> click the "Documents" tab in the blue top-nav bar.

tinagueth commented 1 year ago

@smclaughlin7 Thank you. I noticed that I have the PDS Common Data Dictionary bookmarked (though not the most current one, i.e. 1.19) but not the main page. I appreciate it.

c-suh commented 9 months ago

Resubmitted in https://github.com/NASA-PDS/operations/issues/400.