Migrate OwP Objects from MS 1981 into UAT (PID 35)

yalelibrary / YUL-DC

Preliminary issue tracking for Yale University Libraries Digital Collections project

3 stars 0 forks source link

Migrate OwP Objects from MS 1981 into UAT (PID 35) #2814

Open sshetenhelm opened 2 months ago

sshetenhelm commented 2 months ago

Story Migrate 6,840 parent objects from the Kissinger MS 1981 collection (PID 35) from FindIt to DCS. Parents OIDs in this attached file:

MS1981-ForMigration-OwP.csv

Total objects with Public should be ~12,603.

Acceptance

[x] Export image files from Fedora
[x] Import image files into the DCS pairtree
[x] Generate list of Parent OIDs (use list above)
[ ] Import the objects into DCS UAT into the MSSA admin set as Private
[ ] Import OCR files into DCS and update full text status
[ ] Audit images
[ ] Audit OCR
[ ] Audit parents for ArchivesSpace information
[ ] Create backlog ticket for PROD migration

DraxIndustries79 commented 2 months ago

Waiting for downloading step.

MaggieZhaoYale commented 1 month ago

Copying the images files from the share to DCS pairtree, got this : Linux Errno::EHOSTDOWN: Host is down - sendfile yesterday and today. It looks that the target pairtree is not responding or unreachable. Checking with Rick.

From Rick: on-going Equinix issue (network infrastructure between Yale and Amazon) may cause this.

MaggieZhaoYale commented 4 weeks ago

List of parents OIDs

MS1981_ForMigration_mssa.csv

sshetenhelm commented 4 weeks ago

First batch process here - https://collections-uat.library.yale.edu/management/batch_processes/2016

Seeing if these 998 work before doing thousands and thousands :)

sshetenhelm commented 3 weeks ago

Attempting the next batch process - https://collections-uat.library.yale.edu/management/batch_processes/2033

sshetenhelm commented 2 weeks ago

214 out of 250 failures, PTIFF "expected file not found" happening again - example

martinlovell commented 2 weeks ago

I checked a few random failed ones, and it does not look like the files are in /data.

@K8Sewell

MaggieZhaoYale commented 2 weeks ago

@sshetenhelm I will re-run the script for the failed parents.

MaggieZhaoYale commented 1 week ago

@sshetenhelm Reprocessed the 213 parents https://collections-uat.library.yale.edu/management/batch_processes/2054

sshetenhelm commented 1 week ago

Currently:

449 OwP objects in UAT
~103 of these appear in problem report

Will investigate

sshetenhelm commented 1 week ago

@MaggieZhaoYale here are the results from the problem report - 103 parent objects are missing one or more child image: MS1981-Broken-ParentChild.csv

And here is the batch process from our chat - Additional 10 parents failed this morning: https://collections-uat.library.yale.edu/management/batch_processes/2060

sshetenhelm commented 6 days ago

The 10 re-processed parents look great!

I'll try the first batch of 500 this afternoon

sshetenhelm commented 4 days ago

All 500 failed for no PTIFFs.

@MaggieZhaoYale could you please start the script for the images for these 500 parents?

MS1981-500-01.csv

[ ] Ingested images for these 500 parents