This epic address the tiered rollout of some interrelated changes to functionality around resource complete state and quality assurance.
Problems we seek to address:
Figgy's concept of metadata review doesn't seem to match the actual workflow that ingesters use. The complete state is used to make a resource available to users, whether or not the resource needs additional work.
If a resource is complete before derivatives are generated, it is preserved initially, then preserved again as each derivative is generated (because of the metadata update on the file set). This will happen when resources are ingested directly into the complete state.
Completing a resource mints an ARK. When there's an ingest error, staff users generally delete the resource and ingest again. Combining this with ingest directly into the complete state means unnecessarily minting ARKs.
Making changes to a resource that's already been marked complete results in additional preservation actions.
When a resource has been marked complete before all processing is finished, there may be bad files that aren't caught until they result in errors when users go to access the resource (e.g., errors in pdf generation). This means we get the errors through honeybadger and spend developer time doing quality assurance work on ingested resources, or identifying the curator who can do that work.
We'd like to make changes to the timing of resource completion in a way that will not inconvenience staff users. We'd also like to provide more information to staff users about resource health so that they can identify and repair issues with resources without requiring developer time.
Implementation
Tiered roll out:
[x] #5579 Add “complete when done” to bulk ingest (ready)
[x] #5580 improve messaging so users know what files need to be remediated to be a happy resource. (ready)
[x] #3592 Fix the too many preservation objects bug. (in progress)
[x] #5566 Remove complete state from bulk ingest options (Inbox)
[x] #5581 Block moving to complete if derivatives aren’t good. (Inbox)
This epic address the tiered rollout of some interrelated changes to functionality around resource complete state and quality assurance.
Problems we seek to address:
We'd like to make changes to the timing of resource completion in a way that will not inconvenience staff users. We'd also like to provide more information to staff users about resource health so that they can identify and repair issues with resources without requiring developer time.
Implementation
Tiered roll out:
For even more context see notes in https://docs.google.com/document/d/1r2ektkqUVkTlY2EW1Tl-UaM1J513s_2GaWqrDbHvnvs/edit#