search

NASA-PDS / validate

Validates PDS4 product labels, data and PDS3 Volumes
https://nasa-pds.github.io/validate/
Apache License 2.0
16 stars 11 forks source link

validate does not report missing bundle/collection members #288

Closed tbarnes4 closed 3 years ago

tbarnes4 commented 3 years ago

Is your feature request related to a problem? Please describe. When validating a PDS4 bundle or collection, there might sometimes be a file not copied over, or a file that has the wrong LID/LIDVID or an erroneous LID/LIDVID was included. I don't know if there is another tool that would do this lets say, reference integrity, or not. To know if a bundle.xml or collection.xml validly report's it's members, especially their primary members, would be very helpful in generating and validating these data volumes.

Describe the solution you'd like The validate tool for PDS3 will check the index files to confirm all data files for a data set are present and report when something is missing. The PDS4 side of the tool does not appear to do this. I would ask that the PDS4 side will report if for a bundle, it checks to see if the collection exists, and if not to report it (either online via the registry or offline). Similarly I would ask that for a collection, each product listed in the inventory.csv file be checked for inclusion and reported back when they are missing.

Describe alternatives you've considered Add this feature as a flag. Additionally add a flag for online vs offline.

Additional context The pds-deep-archive tool requires that a bundle pass the validate tool and be complete. If there are no checks to see if the bundle being wrapped up for ingestion is complete, we work on faith that they are.

jordanpadams commented 3 years ago

@tbarnes4 please see https://github.com/NASA-PDS/validate/issues/287#issuecomment-767868233 . I believe that should solve this issue.

tbarnes4 commented 3 years ago

@jordanpadams See my #287. Your suggestion partially resolves the issue, but it appears the real issue is with my example bundle and many labels missing schematrons.

Without the -R flag, in what cases will the validate tool "auto-detect" a pds4.bundle or pds4.collection?

This question aside, I think can close this issue. Thanks for the assistance.

jordanpadams commented 3 years ago

@tbarnes4 we should remove that description from the target flag. It does not accurately auto-detect the different rules. The rule should be specified wherever possible

tbarnes4 commented 3 years ago

@jordanpadams Should I register a documentation issue for the target flag and rule flag?

jordanpadams commented 3 years ago

@tbarnes4 that would be great. Thanks!