search

NASA-PDS / harvest

Standalone Harvest client application providing the functionality for capturing and indexing product metadata into the PDS Registry system (https://github.com/nasa-pds/registry).
https://nasa-pds.github.io/registry
Other
4 stars 3 forks source link

Harvest not finding collections >1 level deep in a directory structure #122

Closed jordanpadams closed 1 year ago

jordanpadams commented 1 year ago

Checked for duplicates

Yes - I've already checked

🐛 Describe the bug

When I tried to harvest LRO diviner data, where the collections are 2 levels deep below the bundle, harvest does not sufficiently crawl the directories to find the data.

🕵️ Expected behavior

I expected harvest would continue crawling and get all files below the parent directory.

📜 To Reproduce

$ harvest -c /path/to/config.xml

image

🖥 Environment Info

📚 Version of Software Used

Question out to user

🩺 Test Data / Additional context

Data RDR - https://pds-geosciences.wustl.edu/lro/lro-l-dlre-4-rdr-v1/ EDR - https://pds-geosciences.wustl.edu/lro/lro-l-dlre-2-edr-v1/

Config Waiting for response from user

🦄 Related requirements

No response

⚙️ Engineering Details

No response

alexdunnjpl commented 1 year ago

self-note: requested user test with both <bundles> and <directories> (and with latest harvest version)

alexdunnjpl commented 1 year ago

@jordanpadams I decided to test it as I already have something set up and it's quick.

Confirmed working with latest state of main branch, using both <bundles> and <directories> approaches.

jordanpadams commented 1 year ago

@alexdunnjpl copy. hopefully this is an issue with the version being used, and not an issue with the software on Windows...

jordanpadams commented 1 year ago

the issue was with the configuration not including directory paths, once the <directories> config was specified, it works just fine.