Added or deleted children not being synced from Preservica

[motropuk]

Story

While testing #2039, it was discovered that new or deleted children in Preservica were not being picked up by DCS

• When using the Digipres-IngestSingleFile workflow in Preservica to add a new file, the file is added as expected. However, when looking at the XML, the asset is missing a representation name (probably not an issue once #2079 and #2072 is implemented). It also has an SHA1 fixity algorithm, when DCS expects SHA-256 or SHA-512. Both of these issues could be causing the sync to not work for new children.

• However, while these issues are likely causing the resync with Preservica to not pickup the new child object, the sync with Preservica batch process also does not appear to be doing anything/not completing, whenever a change is detected in Preservica. For jobs where no change is detected, e.g https://collections-uat.library.yale.edu/management/batch_processes/1020, you get a message, “Child object count and order is the same. No update needed.” However, for other jobs where changes (appear) to be detected, the job just seems to run and not do anything, e.g https://collections-uat.library.yale.edu/management/batch_processes/1049 and https://collections-uat.library.yale.edu/management/batch_processes/1053

Acceptance

• [x] Digipres-IngestSingleFile adds child to preservica with an SHA-512 fixity algorithm
• [x] Resync with Preservica batch job picks up new child objected as expected from Preservica and creates a new child in DCS
• [x] Resync with Preservica batch job recognizes deleted child object in Preservica and deletes existing child in DCS

[motropuk]

Have asked DPS to update the Digipres-IngestSingleFile workflow in Preservica to use SHA-512

[motropuk]

From DPS

It seems like SHA is set somewhere as the default, once I can track it down we can changes it sha512, I may need input from preservica

In the mean time, after ingest ingest, either in the actions menu for the asset or by right clicking on the asset and choosing actions, there is an add fixity workflow, this will add sha512 to the file.

[motropuk]

Error which might explain why sync job did not fail Error detected in: yul-dc-management (production) Full report at https://app.honeybadger.io/projects/72674/faults/85424630

NameError: uninitialized constant #::PreservicaImageServiceNetworkError

Summary SyncFromPreservicaJob#perform

Hostname: ip-10-5-69-200.ec2.internal

Context component: SyncFromPreservicaJob action: perform job_id: 9028117 handler: --- !ruby/object:ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper job_data: job_class: SyncFromPreservicaJob ... last_error: attempts: 0 queue: default Trace app/models/batch_process.rb:404:in rescue in block in sync_from_preservica app/models/batch_process.rb:394:in block in sync_from_preservica app/models/batch_process.rb:379:in each_with_index Reply to this email to comment. You can also reply with @takeit, @assign [teammate's name or email], @resolve, @unsubscribe, or @ignore.

[motropuk]

Manually updated fixity to child added to "4bb9ccab-0f9f-4f6c-93ba-d0b7428bd971" in Preservica, to SHA-512. Ran resync again. Still not working https://collections-uat.library.yale.edu/management/batch_processes/1032. So while we need to fix the fixty issue in Preservica, its not the only thing causing this resync to not work.

[motropuk]

Adding myself to this to test after #2079 and #2072 are done and this ticket is unblocked

[motropuk]

First sync to pickup a new child from Preservica, created the child record in DCS, but did not bring over the image https://collections-uat.library.yale.edu/management/batch_processes/1074. This might be because the image in Preservica had a SHA1 fixity value. DCS needs SHA512. So I fixed this in Preservica and reran the sync. But this sync finished as it said all children were accounted for, as the child record had been added in the previous sync https://collections-uat.library.yale.edu/management/batch_processes/1075. So I deleted the child object in DCS and am trying another resync

[motropuk]

Rsync to pick up added child completed, but only after click on Update this Item on newly created child record. Creating a ticket to update the code to make sure it triggers the “Update this item” process on any parents that get modified by a preservica sync.

[motropuk]

Created #2105

[motropuk]

I dont think think the delete child rsync is working https://collections-uat.library.yale.edu/management/batch_processes/1082/parent_objects/900140256. I deleted child 135 in Preservica here, but the child count for the parent is not going down and again the job does not seem to be moving. Child has not been removed https://collections-uat.library.yale.edu/management/child_objects/900140665. @martinlovell will investigate

[motropuk]

Martin created #2109 which might be causing deleted child resync to fail

[motropuk]

Adding 3 for investigation work. Dev work was split into other tickets