This is perhaps a minor problem and may have some rationale I'm not aware of, but I want to describe it here because it may have more significance shortly as we introduce some changes in the Jisc project.
The job "Verify Bag Compliance" occurs as part of the Approve Transfer microservice. If you create a transfer with a normal directory but select the transfer type "Unzipped Bag", the job "Verify Bag Compliance runs and fails, which then invokes the Failed Transfer microservice.
This behavior is different to the other compliance jobs that run within the "Verify Transfer Compliance" microservice. If these jobs fail, a different microservice is invoked called "Failed Transfer Compliance", and the user is given 3 choices - to reject the transfer, to attempt a restructure for compliance (an automated task that AM will attempt) or move the transfer back to the ActiveTransfers watched directory. (this last option presumably is there to give the user time to manually fix the transfer in the filesystem directly if they want to)
This inconsistency has probably never been noticed because until now, the only way for "Verify Transfer Compliance" to fail is if one of the Archivematica jobs fails (creating the objects/logs/metadata directories). I don't see any ability for a user to make this job fail on purpose.
However, we are about to introduce a change to rdss-archivematica that makes "Verify Transfer Compliance" check that there is at least one file in a transfer. Once that is complete, it can (and I believe should be) merged upstream to the public project. See Add checks for empty transfers.
Another reason we may want to address this is because of the work being done to make the 'start transfer' process asynchronous. See PR artefactual/archivematica#936. This PR removes the need to use a watched folder to initiate a transfer, which makes the initial step in "approve transfer" redundant.
The exception is for Bags... if you test the branch with PR artefactual/archivematica#936, you'll see that for Standard, Dspace or Disk Image transfer types, the first Microservice that gets invoked is "Verify Transfer Compliance".
But in the case where Transfer Type = Bag the first job invoked is "Rename with transfer UUID" (part of Approve Transfer Microservice). Where Transfer Type = Unzipped Bag, the first job invoked is "Extract zipped bag transfer" (also part of the "Approve Transfer" microservice).
so apologies if I'm being longwinded. To sum up, I propose that jobs "Verify Bag Compliance", "Rename with transfer UUID" and "Extract Zipped bag transfer" should all be moved to the microservice group "Verify Transfer Compliance". That will make the behaviour of compliance failures consistent, and when artefactual/archivematica#936 is implemented, it will make the entire "Approve Transfer" microservice redundant (so it can actually be removed entirely).
ross-spencer
commented
3 years ago
This is perhaps a minor problem and may have some rationale I'm not aware of, but I want to describe it here because it may have more significance shortly as we introduce some changes in the Jisc project.
The job "Verify Bag Compliance" occurs as part of the Approve Transfer microservice. If you create a transfer with a normal directory but select the transfer type "Unzipped Bag", the job "Verify Bag Compliance runs and fails, which then invokes the Failed Transfer microservice.
This behavior is different to the other compliance jobs that run within the "Verify Transfer Compliance" microservice. If these jobs fail, a different microservice is invoked called "Failed Transfer Compliance", and the user is given 3 choices - to reject the transfer, to attempt a restructure for compliance (an automated task that AM will attempt) or move the transfer back to the ActiveTransfers watched directory. (this last option presumably is there to give the user time to manually fix the transfer in the filesystem directly if they want to)
This inconsistency has probably never been noticed because until now, the only way for "Verify Transfer Compliance" to fail is if one of the Archivematica jobs fails (creating the objects/logs/metadata directories). I don't see any ability for a user to make this job fail on purpose.
However, we are about to introduce a change to rdss-archivematica that makes "Verify Transfer Compliance" check that there is at least one file in a transfer. Once that is complete, it can (and I believe should be) merged upstream to the public project. See Add checks for empty transfers.
Another reason we may want to address this is because of the work being done to make the 'start transfer' process asynchronous. See PR artefactual/archivematica#936. This PR removes the need to use a watched folder to initiate a transfer, which makes the initial step in "approve transfer" redundant.
The exception is for Bags... if you test the branch with PR artefactual/archivematica#936, you'll see that for Standard, Dspace or Disk Image transfer types, the first Microservice that gets invoked is "Verify Transfer Compliance".
But in the case where Transfer Type = Bag the first job invoked is "Rename with transfer UUID" (part of Approve Transfer Microservice). Where Transfer Type = Unzipped Bag, the first job invoked is "Extract zipped bag transfer" (also part of the "Approve Transfer" microservice).
so apologies if I'm being longwinded. To sum up, I propose that jobs "Verify Bag Compliance", "Rename with transfer UUID" and "Extract Zipped bag transfer" should all be moved to the microservice group "Verify Transfer Compliance". That will make the behaviour of compliance failures consistent, and when artefactual/archivematica#936 is implemented, it will make the entire "Approve Transfer" microservice redundant (so it can actually be removed entirely).