search

NASA-PDS / deep-archive

PDS Open Archival Information System (OAIS) utilities, including Submission Information Package (SIP) and Archive Information Package (AIP) generators
https://nasa-pds.github.io/deep-archive/
Other
7 stars 4 forks source link

AIP Generator outputs duplicate records in the checksum / transfer manifest #71

Closed jordanpadams closed 4 years ago

jordanpadams commented 4 years ago

Describe the bug When testing with /data/local/starbase/data/pds4/test-bundles/urn-nasa-pds-insight_documents/bundle_insight_documents.xml

To Reproduce

pds-deep-archive -s PDS_GEO -b  https://pds.nasa.gov/data/pds4/test-bundles/ /data/local/starbase/data/pds4/test-bundles/urn-nasa-pds-insight_documents/bundle_insight_documents.xml

Expected behavior The generated AIP contains 106 records, several of which are duplicated, when it should only contain 75 (there are only 75 files in the directory). I imagine this may have something to do with the multiple versions of the collection? the problem only exists for this bundle. all the other test cases (under test-bundles) checkout as expected.

$ wc -l *.tab
   145 compil-comet_v4.0_checksum_manifest_v4.0_20200701.tab
   145 compil-comet_v4.0_sip_v1.0_20200701.tab
   145 compil-comet_v4.0_transfer_manifest_v4.0_20200701.tab
    19 epoxi_mri_test_v1.0_checksum_manifest_v1.0_20200701.tab
    19 epoxi_mri_test_v1.0_sip_v1.0_20200701.tab
    19 epoxi_mri_test_v1.0_transfer_manifest_v1.0_20200701.tab
    19 epoxi_mri_v1.0_checksum_manifest_v1.0_20200701.tab
    19 epoxi_mri_v1.0_sip_v1.0_20200701.tab
    19 epoxi_mri_v1.0_transfer_manifest_v1.0_20200701.tab
   106 insight_documents_v2.0_checksum_manifest_v2.0_20200701.tab
    75 insight_documents_v2.0_sip_v1.0_20200701.tab
   106 insight_documents_v2.0_transfer_manifest_v2.0_20200701.tab
   173 soho_v1.0_checksum_manifest_v1.0_20200701.tab
   173 soho_v1.0_sip_v1.0_20200701.tab
   173 soho_v1.0_transfer_manifest_v1.0_20200701.tab

Applicable requirements :unicorn: #45

nutjob4life commented 4 years ago

📝 @nutjob4life note to self: you can find /data/local/starbase/data/pds4/test-bundles/urn-nasa-pds-insight_documents/bundle_insight_documents.xml on pds-gamma.jpl.nasa.gov