NASA-PDS / harvest

Standalone Harvest client application providing the functionality for capturing and indexing product metadata into the PDS Registry system (https://github.com/nasa-pds/registry).
https://nasa-pds.github.io/registry
Other
4 stars 3 forks source link

As a user, I want to exclude files from harvest run based upon PDS4 product class #219

Open jordanpadams opened 1 week ago

jordanpadams commented 1 week ago

Checked for duplicates

Yes - I've already checked

🧑‍🔬 User Persona(s)

Node Operator

💪 Motivation

...so that I can ignore specific product classes that I do not want loaded into the Registry

📖 Additional Details

From user:

We have non-archive files in our service directories to support serving the data. Things like DOI landing pages, for example, and pre-made bulk download files. The the loader is going to assume that everything in a directory is either a PDS4 label or something pointed to by a PDS4 label, it's going to choke. We could make an exclusion list of directory names, file names, and file extensions to ignore, if that would help.

Acceptance Criteria

Given a root/ directory containing 2 XML products within its sub-directory structure, 1 Product_Observational product, urn:nasa:pds:observational_x, and one Product_Browse product, urn:nasa:pds:browse_y When I perform harvest run with dataPath = root/ and productFilter = Product_Browse Then I expect urn:nasa:pds:observational_x to be loaded into the Registry, and urn:nasa:pds:browse_y to be ignored.

⚙️ Engineering Details

No response

🎉 I&T

No response