Ancestry

[alexdunnjpl]

🗒️ Summary

• Adds utilities for product identifiers
• Adds an ancestry sweeper for populating parent collection/bundle metadata
• Adds the ancestry sweeper to sweepers_driver.py
• Adds functional tests for ancestry sweeper

⚙️ Test Data and/or Report

Ancestry sweeper needs tests as part of I&T suite (as does provenance)

Functional tests implemented/pass for ancestry sweeper Unit tests added for pds.registrysweepers.utils.productidentifiers

♻️ Related Issues

fixes #14

supports https://github.com/NASA-PDS/registry-api/issues/318 supports https://github.com/NASA-PDS/registry-api/issues/319

Some TODOs/minor-features outstanding:

• [x] Add support for alternate_ids
• [x] Add test-cases for alternate_ids
• [x] Add informational logging for the various stages of ancestry processing
• [ ] ~Add search and warn-log for orphaned non-aggregate products~

@jordanpadams if that last one isn't actually important/useful, let me know.

[alexdunnjpl]

converting back to draft to ensure that outstanding minor items in OP aren't overlooked when I come back to this next week

[alexdunnjpl]

@tloubrieu-jpl ready for review

[alexdunnjpl]

@jordanpadams see https://github.com/NASA-PDS/registry-sweepers/pull/10/commits/95489170c96467974272a813e641d3047f04c37a for implementation and test-cases for support of alternative_ids

[tloubrieu-jpl]

@alexdunnjpl I removed the sprint-backlog tag from the PR, since this is bugging me in the zenhub board.

[alexdunnjpl]

Mostly not critical comments from me, but sonatype and Sean's comments require some changes .

@tloubrieu-jpl what of Sean's requires further action?

Regarding Sonatype, are you aware of a way to trigger a fresh sweep of a PR? I think possibly the force pushes are confusing it. Or are you just talking about the one issue mentioned in your recent comment?

[alexdunnjpl]

@tloubrieu-jpl regarding the outstanding sonatype error, ~it's a false positive - not picked up by mypy either.~ actually it's stale - that line is correctly annotated since 6c38aa5

I've removed some obsolete mypy ignores, but otherwise it should be good to go.

[alexdunnjpl]

Just realised there's one outstanding question for @jordanpadams. What exactly do you want the metadata keys to be?

Currently it's ops:Provenance/ops:parent_collection_identifiers and ops:Provenance/ops:parent_bundle_identifiers, but the pluralisation, while arguably correct, may be at odds with user expectations given fields like pds:Internal_Reference/pds:lid_reference which are collections but whose field name is singular.

[jordanpadams]

@alexdunnjpl let's go with the singular

[tloubrieu-jpl]

Mostly not critical comments from me, but sonatype and Sean's comments require some changes .

@tloubrieu-jpl what of Sean's requires further action?

@alexdunnjpl I think Sean requested a change for this comment https://github.com/NASA-PDS/registry-sweepers/pull/10/files#r1192418906

[tloubrieu-jpl]

No sorry Sean said it does not need change. Let's validate the PR