search

archivematica / Issues

Issues repository for the Archivematica project
GNU Affero General Public License v3.0
16 stars 1 forks source link

Problem: Cannot approve dspace file-only transfers in Archivematica 1.8 #468

Closed ross-spencer closed 5 years ago

ross-spencer commented 5 years ago

Expected behaviour

When I start a DSpace transfer from the API and that transfer is file-only transfer I expect to be able to approve it via the API as well.

Current behaviour

This code introduced in stable/1.8 means that a slash is appended to the file-path when a transfer is started if it isn't a zipped_bag. Dspace transfers that might just be a zip file will always have a slash appended.

Tools using the API might look for a path like:

/var/archivematica/sharedDirectory/watchedDirectories/activeTransfers/Dspace/api_test_2.zip

But Archivematica will look for:

/var/archivematica/sharedDirectory/watchedDirectories/activeTransfers/Dspace/api_test_2.zip/

when attempting to approve the transfer. It will not be possible to approve the transfer.

Steps to reproduce

Attempt to start and approve a DSpace transfer from just a zip file in Archivematica using the API.

Your environment (version of Archivematica, OS version, etc)

qa/1.x https://github.com/artefactual/archivematica/commits/qa/1.x but this is also the anticipated behavior in stable/1.8.

For Artefactual use: Please make sure these steps are taken before moving this issue from Review to Verified in Waffle:

sallain commented 5 years ago

Triaged for 1.9.1. as this issue is blocking upgrades for customers who use DSpace + automation tools

ross-spencer commented 5 years ago

When we configure automation tools to transfer dspace files (not directories) we see the following problem with approval:

INFO      2019-03-19 18:38:37 transfer.py:444  Failed transfer approval, try 1 of 3
INFO      2019-03-19 18:38:37 transfer.py:461  Approving uuid_test_2.zip
DEBUG     2019-03-19 18:38:43 utils.py:36   URL: http://127.0.0.1:62080/api/transfer/unapproved; params: {u'username': 'test', u'api_key': 'test'}; method: GET
DEBUG     2019-03-19 18:38:43 utils.py:48   Response: <Response [200]>
DEBUG     2019-03-19 18:38:43 utils.py:49   type(response.text): <type 'unicode'> 
DEBUG     2019-03-19 18:38:43 utils.py:51   Response content-type: application/json
INFO      2019-03-19 18:38:43 transfer.py:481  Found waiting transfer: uuid_test_2.zip
DEBUG     2019-03-19 18:38:43 utils.py:36   URL: http://127.0.0.1:62080/api/transfer/approve/; params: {u'directory': 'uuid_test_2.zip', u'type': u'dspace'}; method: POST
DEBUG     2019-03-19 18:38:43 utils.py:48   Response: <Response [500]>
DEBUG     2019-03-19 18:38:43 utils.py:49   type(response.text): <type 'unicode'> 
DEBUG     2019-03-19 18:38:43 utils.py:51   Response content-type: application/json
WARNING   2019-03-19 18:38:43 utils.py:58   POST Request to http://127.0.0.1:62080/api/transfer/approve/ returned 500 INTERNAL SERVER ERROR
DEBUG     2019-03-19 18:38:43 utils.py:59   Response: {"message": "Unable to start the transfer.", "error": true}
ERROR     2019-03-19 18:38:43 transfer.py:491  Error approving transfer: Invalid response from server, check amclient log
INFO      2019-03-19 18:38:43 transfer.py:444  Failed transfer approval, try 2 of 3
INFO      2019-03-19 18:38:43 transfer.py:461  Approving uuid_test_2.zip
DEBUG     2019-03-19 18:38:49 utils.py:36   URL: http://127.0.0.1:62080/api/transfer/unapproved; params: {u'username': 'test', u'api_key': 'test'}; method: GET
DEBUG     2019-03-19 18:38:49 utils.py:48   Response: <Response [200]>
DEBUG     2019-03-19 18:38:49 utils.py:49   type(response.text): <type 'unicode'> 
DEBUG     2019-03-19 18:38:49 utils.py:51   Response content-type: application/json
INFO      2019-03-19 18:38:49 transfer.py:481  Found waiting transfer: uuid_test_2.zip
DEBUG     2019-03-19 18:38:49 utils.py:36   URL: http://127.0.0.1:62080/api/transfer/approve/; params: {u'directory': 'uuid_test_2.zip', u'type': u'dspace'}; method: POST
DEBUG     2019-03-19 18:38:49 utils.py:48   Response: <Response [500]>
DEBUG     2019-03-19 18:38:49 utils.py:49   type(response.text): <type 'unicode'> 
DEBUG     2019-03-19 18:38:49 utils.py:51   Response content-type: application/json
WARNING   2019-03-19 18:38:49 utils.py:58   POST Request to http://127.0.0.1:62080/api/transfer/approve/ returned 500 INTERNAL SERVER ERROR
DEBUG     2019-03-19 18:38:49 utils.py:59   Response: {"message": "Unable to start the transfer.", "error": true}
ERROR     2019-03-19 18:38:49 transfer.py:491  Error approving transfer: Invalid response from server, check amclient log
INFO      2019-03-19 18:38:49 transfer.py:444  Failed transfer approval, try 3 of 3
WARNING   2019-03-19 18:38:49 transfer.py:448  Transfer not approved: uuid_test_2.zip

The change was introduced here: https://github.com/artefactual/archivematica/commit/145df0c1669e49888a7db9d667766ed1e0323f60#diff-60bfea212a0451303313ec7bc09c8770

If we check the following: if transfer is dspace and the object is a file then we should be able to mitigate this issue. Regular transfers and dspace transfers that are directory based should still be handled regularly even if this check doesn't pass.

A PR is already submitted, I just forgot to press go on this comment before submitting it! ;D

ross-spencer commented 5 years ago

Notes from testing on am191bionic:

image

12 directories, 15 files