ArchivePath is prefixed wrongly for Verify

[blankdots]

Expected Behavior

Verify finds the file that was ingested.

Current Behavior

When ingesting a file verify service fails to find the ingested file due to wrong path:

╰─$ docker-compose logs -f verify
Attaching to verify
verify        | [lega.utils.amqp][DEBUG ] (L38) Setting hearbeat to 0
verify        | [lega.utils.amqp][ INFO ] (L50) Getting a connection to broker
verify        | [lega.utils.amqp][DEBUG ] (L51) {'host': 'mq', 'port': 5672, 'virtual_host': '/', 'credentials': <pika.credentials.PlainCredentials object at 0x7fd51b439e80>, 'connection_attempts': 30, 'retry_delay': 10, 'heartbeat_interval': 0}
verify        | [lega.utils.amqp][DEBUG ] (L84) Consuming message from archived
verify        | [lega.utils.amqp][DEBUG ] (L93) Consuming message 1 (Correlation ID: 84306f6e-4b37-11e9-b7d0-0242ac1b0007)
verify        | [lega.verify][ INFO ] (L71) Verification | message: {'user': 'dummy', 'filepath': '/fa43512c-6d5b-41e2-9783-56f3823e2afe.c4ga', 'org_msg': {'user': 'dummy', 'filepath': '/fa43512c-6d5b-41e2-9783-56f3823e2afe.c4ga'}, 'file_id': 1, 'header': '6372797074346768010000009c020000c1c14c033d6da5cbb3022750011000b186716387b5f98ea24abe9def5bb4ebb70387f8d588b99f928254160121a30e8101b89ecfca7644f5731c2074020bb3a3929302b1b0c9e0aaf6cd9b5118dfd4abb3b2bd31e42eea34a7bfd737742145c3a185d4f8cb85bc0aa9f9645828c6374e1629a5df3cc7ce5358573e8e4e95b8960300d8931d1b96749c5081e82062fb0dc92937c0f1812dbf64727be3348f768ea870ff7a76a1311bed1e89e6708ae8373f6e8df6ec8eb0b7f966ab8a1f3fa57ae0b022657ecc1cd012c1b3a38eadc14bf0f16578fb20f30ccc3ed19a288c68090392c69d3bf0794b23261e2310e2f44490c503eb85aae7fc70e65f82150df6ebd524e218391c2179c02f3676365c34f50d05dffa155f1144dd7f8f4e854205aaae7f935404728affddc6eacf879bf340707ea34953b259072a0a2fdc65222dde86821756a41805923255df7e89651fb94f716c9dbe202267f0f865a6c05fbdf21efb19461db252b59fef0f317ee771d9fc30b0d09113ff31f2fc90253da50c4a853a9a65d7a7730d6664988197f96ae1d00c438c2bd11d4be8df66de6ee0bc91717d3822981aca66bd424c216c4b6ff588c990570e4491e856a1820e19edc76c64124238b13c737c9da8083bcf0f31465401c67ffbd540f6a44c10a692a775b1e714734759d5c5e9c53f434958d43f4f8b1e11db523f3fec7d90842294be656c59aa24cd0d5f3d95ae9bd8bd7907b4d27b01fc41757f8144bbca4bb4a0cd607df3d71bea7ca0256b5d826b28acc138c9193dbe1de33c919583d01ee66cf9e4760e70098aaaba0a94d9ebe27e793809b0c4012f530048470a129db672ee200671f66bdf2596c737af1ca6d624594e527889c277f6077810413aa61be9963f92da99fe44c301cb9e608ae52494', 'archive_path': '/ega/archive/lega/000/000/000/000/000/000/01'}
verify        | [lega.verify][ INFO ] (L34) Key ID 3D6DA5CBB3022750
verify        | [lega.verify][ INFO ] (L37) Retrieving the Private Key from http://keys:8080/keys/retrieve/3D6DA5CBB3022750/private/bin?idFormat=hex (verify certificate: False)
verify        | [lega.verify][ INFO ] (L81) Opening archive file: /ega/archive/lega/000/000/000/000/000/000/01
verify        | [lega.utils.db][ERROR ] (L252) Exception: <class 'FileNotFoundError'> in /usr/local/lib/python3.6/site-packages/lega/utils/db.py on line: 283
verify        | [lega.utils.db][ERROR ] (L255) FileNotFoundError(2, 'No such file or directory') (from user: False)
verify        | [lega.utils.db][DEBUG ] (L142) Setting error for 1: [Errno 2] No such file or directory: '/ega/archive/lega/ega/archive/lega/000/000/000/000/000/000/01' | Cause: None
verify        | [lega.utils.db][DEBUG ] (L49) 30 attempts (every 1 seconds)
verify        | [lega.utils.db][ INFO ] (L34) Initializing a connection to: db:5432/lega
verify        | [lega.utils.db][DEBUG ] (L262) Catching error on file id: 1
verify        | [lega.utils.amqp][DEBUG ] (L104) Sending ACK for message 1 (Correlation ID: 84306f6e-4b37-11e9-b7d0-0242ac1b0007)

Notice the '/ega/archive/lega/ega/archive/lega/000/000/000/000/000/000/01' This might be caused by the fact that the in the db the 'archive_path': '/ega/archive/lega/000/000/000/000/000/000/01' is like that and this line adds a prefix: https://github.com/EGA-archive/LocalEGA/blob/master/lega/utils/storage.py#L45

Possible Solution

Either store the archive path differently or remove the prefix when retrieving with verify.

Steps to Reproduce

1. make bootstrap ARGS='--keyserver ega'
2. make the necessary changes for FileStorage driver
   • change in lega.yml in ingest, verify and res the volume to archive:/ega/archive/lega
   • comment out archive service
   • replace archive in conf.ini with

     location = /ega/archive/%s/
     mode = 2750
     storage_driver = FileStorage

3. Comment out the user: lega in docker compose because of this issue: https://github.com/EGA-archive/LocalEGA/issues/45
4. make up
5. cd in lega/tests and run bats _other/simple.bats

Context (Environment)

I have been trying to use Posix as a backend for RES.

Did you prune volumes?

yes, did that.

Was the file ingested?

yes, checked