[data provider issue] UCD multiple records with ARK conflicts (3 pairs of records share ARKs - 6 total records from 3 collections)

[christinklez]

3 UCD collections with ARK conflicts.

Note: These are not harvest-stopping errors; these collections are currently on -stage.

To Do:

• [ ] Contact UCD about the ARK conflicts
• [ ] Reharvest
  • [ ] 5656 ark:/87293/d39444
  • [ ] 28050 ark:/87293/d3db7vt1v
  • [ ] 28052 ark:/87293/d3959cc77

Registry ID: 5656 ark:/87293/d39444

• These two UCD records: https://digital.ucdavis.edu/search/d39444/%5B%5D//10/
• what's on -test: https://calisphere-test.cdlib.org/item/ark:/87293/d39444/ (points to: https://digital.ucdavis.edu/collection/davis-scouts/MC-201/001-Photograph_album_Front_Cover)
• Run ID: manual__2024-04-29T16:23:28+00:00
• Airflow gridview: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/dags/harvest_collection/grid?num_runs=365&dag_run_id=manual__2024-04-29T16%3A23%3A28%2B00%3A00&base_date=2024-04-29T16%3A23%3A28%2B00%3A00
• Permalink to log: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/log?dag_id=harvest_collection&task_id=create_stage_index&execution_date=2024-04-29T16%3A23%3A28%2B00%3A00

Exception: 1 errors in bulk indexing 10 records: ['[ark:/87293/d39444]: version conflict, document already exists (current version [1])']

mapped_metadata page 3 UCD record: https://digital.ucdavis.edu/collection/davis-scouts/MC-201/full_album [{"calisphere-id": "ark:/87293/d39444", "campus_data": ["2::UC Davis"], "campus_name": ["UC Davis"], "campus_url": ["2"], "collection_data": ["5656::Davis Boy Scout Troop No. 1 Photograph Album"], "collection_name": ["Davis Boy Scout Troop No. 1 Photograph Album"], "collection_url": ["5656"], "date": ["1927"], "description": ["Photographs of Davis Boy Scout Troop #1."], "fetcher_type": ["ucd_json"], "id": "ark:/87293/d39444", "identifier": ["ark:/87293/d39444", "MC-201", "/collection/davis-scouts/MC-201/full_album"], "is_shown_at": "https://digital.ucdavis.edu/collection/davis-scouts/MC-201/full_album", "is_shown_by": "https://digital.ucdavis.edu/fcrepo/rest/collection/davis-scouts/MC-201/full_album/media/images/001-Photograph_album_Front_Cover.tif/svc:iiif/full/500,291/0/default.jpg", "item_count": 0, "mapper_type": ["ucd_json.ucd_json"], "media_source": {}, "publisher": ["UC Davis Library, Archives and Special Collections"], "repository_data": ["12::University Library, Special Collections::UC Davis"], "repository_name": ["University Library, Special Collections"], "repository_url": ["12"], "rights_uri": "http://rightsstatements.org/vocab/InC-NC/1.0/", "sort_collection_data": ["davis boy scout troop no 1 photograph album::Davis Boy Scout Troop No. 1 Photograph Album::5656"], "sort_title": "000 full album", "thumbnail_source": {"url": "https://digital.ucdavis.edu/fcrepo/rest/collection/davis-scouts/MC-201/full_album/media/images/001-Photograph_album_Front_Cover.tif/svc:iiif/full/500,291/0/default.jpg"}, "title": ["000 Full Album"], "type": [""], "url_item": "https://digital.ucdavis.edu/collection/davis-scouts/MC-201/full_album", "thumbnail": {"mimetype": "image/jpeg", "path": "s3://rikolti-content/thumbnails/5656/ee9b72bf5db72527eb22ccb6fe08a8ba", "dimensions": [500, 291]}},

mapped_metadata page 4 UCD record: https://digital.ucdavis.edu/collection/davis-scouts/MC-201/001-Photograph_album_Front_Cover [{"calisphere-id": "ark:/87293/d39444", "campus_data": ["2::UC Davis"], "campus_name": ["UC Davis"], "campus_url": ["2"], "collection_data": ["5656::Davis Boy Scout Troop No. 1 Photograph Album"], "collection_name": ["Davis Boy Scout Troop No. 1 Photograph Album"], "collection_url": ["5656"], "date": ["1922"], "description": ["Photographs Davis Troop #1. Boy Scouts of America, Davis, Calif."], "fetcher_type": ["ucd_json"], "id": "ark:/87293/d39444", "identifier": ["ark:/87293/d39444", "MC-201", "/collection/davis-scouts/MC-201/001-Photograph_album_Front_Cover"], "is_shown_at": "https://digital.ucdavis.edu/collection/davis-scouts/MC-201/001-Photograph_album_Front_Cover", "is_shown_by": "https://digital.ucdavis.edu/fcrepo/rest/collection/davis-scouts/MC-201/001-Photograph_album_Front_Cover/001-Photograph_album_Front_Cover.tif/svc:iiif/full/500,291/0/default.jpg", "item_count": 0, "mapper_type": ["ucd_json.ucd_json"], "media_source": {}, "publisher": ["UC Davis Library, Archives and Special Collections"], "repository_data": ["12::University Library, Special Collections::UC Davis"], "repository_name": ["University Library, Special Collections"], "repository_url": ["12"], "rights_uri": "http://rightsstatements.org/vocab/InC-NC/1.0/", "sort_collection_data": ["davis boy scout troop no 1 photograph album::Davis Boy Scout Troop No. 1 Photograph Album::5656"], "sort_title": "001 cover", "thumbnail_source": {"url": "https://digital.ucdavis.edu/fcrepo/rest/collection/davis-scouts/MC-201/001-Photograph_album_Front_Cover/001-Photograph_album_Front_Cover.tif/svc:iiif/full/500,291/0/default.jpg"}, "title": ["001 Cover"], "type": [""], "url_item": "https://digital.ucdavis.edu/collection/davis-scouts/MC-201/001-Photograph_album_Front_Cover", "thumbnail": {"mimetype": "image/jpeg", "path": "s3://rikolti-content/thumbnails/5656/ee9b72bf5db72527eb22ccb6fe08a8ba", "dimensions": [500, 291]}},

Registry ID: 28050 ark:/87293/d3db7vt1v

• These two UCD records: https://digital.ucdavis.edu/search/d3db7vt1v/%5B%5D//10/
• what's on -test: https://calisphere-test.cdlib.org/item/ark:/87293/d3db7vt1v/ (points to: https://digital.ucdavis.edu/collection/lug_labels/D-211/D211_34_116a)
• Run ID: manual__2024-04-29T19:43:00+00:00
• Airflow gridview: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/dags/harvest_collection/grid?num_runs=365&base_date=2024-04-29T19%3A43%3A00%2B00%3A00&dag_run_id=manual__2024-04-29T19%3A43%3A00%2B00%3A00&task_id=create_stage_index&tab=logs
• Permalink to log: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/log?dag_id=harvest_collection&task_id=create_stage_index&execution_date=2024-04-29T19%3A43%3A00%2B00%3A00

Exception: 1 errors in bulk indexing 10 records: ['[ark:/87293/d3db7vt1v]: version conflict, document already exists (current version [1])']

Probably these two UCD records: https://digital.ucdavis.edu/collection/lug_labels/D-211/D211_34_116a https://digital.ucdavis.edu/collection/lug_labels/D-211/D211_34_116

Registry ID: 28052 ark:/87293/d3959cc77

• These two UCD records: https://digital.ucdavis.edu/search/d3959cc77/%5B%5D//10/
• what's on -test: https://calisphere-test.cdlib.org/item/ark:/87293/d3959cc77/ (points to: https://digital.ucdavis.edu/collection/Picnic_Day/AR-075/Picnic_Day_2002)
• Run ID: manual__2024-04-29T19:43:05+00:00
• Airflow gridview: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/dags/harvest_collection/grid?num_runs=365&base_date=2024-04-29T19%3A43%3A05%2B00%3A00&dag_run_id=manual__2024-04-29T19%3A43%3A05%2B00%3A00&task_id=create_stage_index&tab=logs
• Permalink to log: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/log?dag_id=harvest_collection&task_id=create_stage_index&execution_date=2024-04-29T19%3A43%3A05%2B00%3A00

Exception: 1 errors in bulk indexing 10 records: ['[ark:/87293/d3959cc77]: version conflict, document already exists (current version [1])']

mapped metadata page 0 UCD record: https://digital.ucdavis.edu/collection/Picnic_Day/AR-075/Picnic_Day_2001 {"calisphere-id": "ark:/87293/d3959cc77", "campus_data": ["2::UC Davis"], "campus_name": ["UC Davis"], "campus_url": ["2"], "collection_data": ["28052::Picnic Day Collection"], "collection_name": ["Picnic Day Collection"], "collection_url": ["28052"], "creator": ["University of California, Davis. Associated Students"], "date": ["2001"], "fetcher_type": ["ucd_json"], "id": "ark:/87293/d3959cc77", "identifier": ["ark:/87293/d3959cc77", "/collection/Picnic_Day/AR-075/Picnic_Day_2001"], "is_shown_at": "https://digital.ucdavis.edu/collection/Picnic_Day/AR-075/Picnic_Day_2001", "is_shown_by": "https://digital.ucdavis.edu/fcrepo/rest/collection/Picnic_Day/AR-075/Picnic_Day_2001/media/Picnic_Day_program_2001-00.jpg/svc:iiif/full/500,461/0/default.jpg", "item_count": 0, "mapper_type": ["ucd_json.ucd_json"], "media_source": {}, "publisher": ["UC Davis Library, Archives and Special Collections"], "repository_data": ["12::University Library, Special Collections::UC Davis"], "repository_name": ["University Library, Special Collections"], "repository_url": ["12"], "rights_uri": "http://rightsstatements.org/vocab/InC-NC/1.0/", "sort_collection_data": ["picnic day collection::Picnic Day Collection::28052"], "sort_title": "picnic day program 2001", "subject": ["Festivals", "University of California, Davis", "Picnic Day"], "thumbnail_source": "https://digital.ucdavis.edu/fcrepo/rest/collection/Picnic_Day/AR-075/Picnic_Day_2001/media/Picnic_Day_program_2001-00.jpg/svc:iiif/full/500,461/0/default.jpg", "title": ["Picnic Day program 2001"], "type": [""], "url_item": "https://digital.ucdavis.edu/collection/Picnic_Day/AR-075/Picnic_Day_2001"},

mapped metadata page 5 UCD record: https://digital.ucdavis.edu/collection/Picnic_Day/AR-075/Picnic_Day_2002 {"calisphere-id": "ark:/87293/d3959cc77", "campus_data": ["2::UC Davis"], "campus_name": ["UC Davis"], "campus_url": ["2"], "collection_data": ["28052::Picnic Day Collection"], "collection_name": ["Picnic Day Collection"], "collection_url": ["28052"], "creator": ["University of California, Davis. Associated Students"], "date": ["2002"], "fetcher_type": ["ucd_json"], "id": "ark:/87293/d3959cc77", "identifier": ["ark:/87293/d3959cc77", "/collection/Picnic_Day/AR-075/Picnic_Day_2002"], "is_shown_at": "https://digital.ucdavis.edu/collection/Picnic_Day/AR-075/Picnic_Day_2002", "is_shown_by": "https://digital.ucdavis.edu/fcrepo/rest/collection/Picnic_Day/AR-075/Picnic_Day_2002/media/Picnic_Day_2002_001.jpg/svc:iiif/full/452,500/0/default.jpg", "item_count": 0, "mapper_type": ["ucd_json.ucd_json"], "media_source": {}, "publisher": ["UC Davis Library, Archives and Special Collections"], "repository_data": ["12::University Library, Special Collections::UC Davis"], "repository_name": ["University Library, Special Collections"], "repository_url": ["12"], "rights_uri": "http://rightsstatements.org/vocab/InC-NC/1.0/", "sort_collection_data": ["picnic day collection::Picnic Day Collection::28052"], "sort_title": "picnic day program 2002", "subject": ["Festivals", "University of California, Davis", "Picnic Day"], "thumbnail_source": "https://digital.ucdavis.edu/fcrepo/rest/collection/Picnic_Day/AR-075/Picnic_Day_2002/media/Picnic_Day_2002_001.jpg/svc:iiif/full/452,500/0/default.jpg", "title": ["Picnic Day program 2002"], "type": [""], "url_item": "https://digital.ucdavis.edu/collection/Picnic_Day/AR-075/Picnic_Day_2002"},

[christinklez]

5656 [2024-05-10, 21:47:52 UTC] {{logging_mixin.py:150}} INFO - WARNING - document already exists; not creating. {'create': {'_index': 'rikolti-stg-combined-20240508151215', '_id': 'ark:/87293/d39444', 'status': 409, 'error': {'type': 'version_conflict_engine_exception', 'reason': '[ark:/87293/d39444]: version conflict, document already exists (current version [1])', 'index': 'rikolti-stg-combined-20240508151215', 'shard': '0', 'index_uuid': '11KZILPMsrT8eb863fOrk8-Q'}}} On -stage: https://calisphere-stage.cdlib.org/item/ark:/87293/d39444/ Points to: https://digital.ucdavis.edu/collection/davis-scouts/MC-201/full_album

28050 [2024-05-10, 21:47:23 UTC] {{logging_mixin.py:150}} INFO - WARNING - document already exists; not creating. {'create': {'_index': 'rikolti-stg-combined-20240508151215', '_id': 'ark:/87293/d3db7vt1v', 'status': 409, 'error': {'type': 'version_conflict_engine_exception', 'reason': '[ark:/87293/d3db7vt1v]: version conflict, document already exists (current version [1])', 'index': 'rikolti-stg-combined-20240508151215', 'shard': '0', 'index_uuid': '11KZILPMsrT8eb863fOrk8-Q'}}} On -stage: https://calisphere-stage.cdlib.org/item/ark:/87293/d3db7vt1v Points to: https://digital.ucdavis.edu/collection/lug_labels/D-211/D211_34_116

28052 [2024-05-10, 21:49:50 UTC] {{logging_mixin.py:150}} INFO - WARNING - document already exists; not creating. {'create': {'_index': 'rikolti-stg-combined-20240508151215', '_id': 'ark:/87293/d3959cc77', 'status': 409, 'error': {'type': 'version_conflict_engine_exception', 'reason': '[ark:/87293/d3959cc77]: version conflict, document already exists (current version [1])', 'index': 'rikolti-stg-combined-20240508151215', 'shard': '0', 'index_uuid': '11KZILPMsrT8eb863fOrk8-Q'}}} On -stage: https://calisphere-stage.cdlib.org/item/ark:/87293/d3959cc77/ Points to: https://digital.ucdavis.edu/collection/Picnic_Day/AR-075/Picnic_Day_2001