search

EGA-archive / LocalEGA

A federated storage for sensitive data
http://localega.readthedocs.io
Apache License 2.0
7 stars 16 forks source link

LocalEGA testsuite not working as expected on my LocalEGA testkit #134

Closed claudiologiudice closed 8 months ago

claudiologiudice commented 10 months ago

Expected Behavior

I'm using the LocalEGA testsuite

Current Behavior

It shoud accomplish all the selected tasks, according to a fake central ega node.

Steps to Reproduce

bats tests/integration ✓ Ingest a file for a user that does not exist in CentralEGA ✓ Ingest a file using the wrong user password ✓ Ingest a file using the wrong user sshkey ✗ Upload a batch of files (from function load_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from functionsetup' in test file tests/integration/inbox.bats, line 24) load_into_ssh_agent ${TESTUSER}' failed ✗ Ingestion of ghost file goes to error (from functionload_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from function setup' in test file tests/integration/inbox.bats, line 24) load_into_ssh_agent ${TESTUSER}' failed ✗ File not found goes to error (from function load_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from functionsetup' in test file tests/integration/inbox.bats, line 24) load_into_ssh_agent ${TESTUSER}' failed ✗ Ingest properly a 10MB file (from functionload_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from function setup' in test file tests/integration/ingestion.bats, line 27) load_into_ssh_agent ${TESTUSER}' failed ✗ Ingest properly a 1GB file (from function load_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from functionsetup' in test file tests/integration/ingestion.bats, line 27) load_into_ssh_agent ${TESTUSER}' failed ✗ Do not ingest the same file twice (from functionload_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from function setup' in test file tests/integration/ingestion.bats, line 27) load_into_ssh_agent ${TESTUSER}' failed ✗ Do not ingest a file not in Crypt4GH format (from function load_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from functionsetup' in test file tests/integration/ingestion.bats, line 27) load_into_ssh_agent ${TESTUSER}' failed ✗ Ingest a file from a subdirectory (from functionload_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from function setup' in test file tests/integration/ingestion.bats, line 27) load_into_ssh_agent ${TESTUSER}' failed ✗ Do not ingest file destined for another LocalEGA (from function load_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from functionsetup' in test file tests/integration/ingestion.bats, line 27) load_into_ssh_agent ${TESTUSER}' failed ✗ Ingest properly a test file, with a dataset mapping (from functionload_into_ssh_agent' in file tests/integration/../_common/helpers.bash, line 85, from function setup' in test file tests/integration/mapping.bats, line 27) load_into_ssh_agent ${TESTUSER}' failed

13 tests, 10 failures

silverdaz commented 10 months ago

Have you tested first if everything was up and running? ...and connected?

Can you please connect to the inbox container, manually, with

$ docker-compose exec inbox bash

and try a command like

$ id john      # you look for the user "john" (used by the testsuite and from the fake CentralEGA)

If the user is not found, you won't be able to connect and even less to run the testsuite.

claudiologiudice commented 9 months ago

id john uid=25000(john) gid=1000(lega) groups=1000(lega)

It seems the user is found, but the testsuite keeps giving me that error...

claudiologiudice commented 9 months ago

I add more information about the problem I'm facing with the fega toolkit: docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 47d547fb26e6 egarchive/lega-inbox:latest "entrypoint.sh" 3 seconds ago Up 3 seconds 0.0.0.0:2222->9000/tcp, :::2222->9000/tcp inbox 3bd76b952351 egarchive/lega-mq:latest "/usr/local/bin/ega-…" 5 seconds ago Up 3 seconds 4369/tcp, 5671-5672/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:15672->15672/tcp, :::15672->15672/tcp mq c7e7e7cab165 rabbitmq:3.9.11-management-alpine "/usr/local/bin/cega…" 5 seconds ago Up 4 seconds 4369/tcp, 5672/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:5670->5671/tcp, :::5670->5671/tcp, 0.0.0.0:15670->15672/tcp, :::15670->15672/tcp cega-mq 1599741d8a85 egarchive/lega-db:latest "/usr/local/bin/entr…" 5 seconds ago Up 4 seconds 0.0.0.0:15432->5432/tcp, :::15432->5432/tcp archive-db 8000a7b38440 egarchive/lega-base:latest "lega-entrypoint.sh …" 5 seconds ago Up 3 seconds ingest d687ad586df9 egarchive/lega-base:latest "lega-entrypoint.sh …" 5 seconds ago Up 3 seconds save2db 61714d6a0724 egarchive/lega-base:latest "lega-entrypoint.sh …" 5 seconds ago Up 4 seconds backup2 ddeee1ca02e1 egarchive/cega-users:latest "python /cega/server…" 5 seconds ago Exited (1) 4 seconds ago cega-users 157f2550ded1 egarchive/lega-db:latest "/usr/local/bin/entr…" 5 seconds ago Up 4 seconds 5432/tcp db 7b89fff20eb3 egarchive/lega-base:latest "lega-entrypoint.sh …" 5 seconds ago Up 3 seconds backup1 5f0bb7d00cc8 egarchive/lega-base:latest "lega-entrypoint.sh …" 5 seconds ago Up 4 seconds cleanup 9aafa91ee6dd egarchive/lega-base:latest "lega-entrypoint.sh …" 5 seconds ago Up 4 seconds dispatcher 9dedc43edcf6 egarchive/lega-base:latest "python accession.py" 5 seconds ago Up 4 seconds cega-accession feganode@fega-ba:/mnt/data/LocalEGA/deploy$ docker logs ddeee1ca02e1 unhandled exception during asyncio.run() shutdown task: <Task finished name='Task-1' coro=<_run_app() done, defined at /usr/local/lib/python3.9/site-packages/aiohttp/web.py:303> exception=TypeError('The port is required to be int.')> Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/aiohttp/web.py", line 544, in run_app loop.run_until_complete(main_task) File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete return future.result() File "/usr/local/lib/python3.9/site-packages/aiohttp/web.py", line 451, in _run_app names = sorted(str(s.name) for s in runner.sites) File "/usr/local/lib/python3.9/site-packages/aiohttp/web.py", line 451, in names = sorted(str(s.name) for s in runner.sites) File "/usr/local/lib/python3.9/site-packages/aiohttp/web_runner.py", line 112, in name return str(URL.build(scheme=scheme, host=host, port=self._port)) File "/usr/local/lib/python3.9/site-packages/yarl/_url.py", line 237, in build raise TypeError("The port is required to be int.") TypeError: The port is required to be int. Traceback (most recent call last): File "/cega/server.py", line 105, in web.run_app(server, host=host, port=port, shutdown_timeout=0, ssl_context=ssl_ctx) File "/usr/local/lib/python3.9/site-packages/aiohttp/web.py", line 544, in run_app loop.run_until_complete(main_task) File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete return future.result() File "/usr/local/lib/python3.9/site-packages/aiohttp/web.py", line 451, in _run_app names = sorted(str(s.name) for s in runner.sites) File "/usr/local/lib/python3.9/site-packages/aiohttp/web.py", line 451, in names = sorted(str(s.name) for s in runner.sites) File "/usr/local/lib/python3.9/site-packages/aiohttp/web_runner.py", line 112, in name return str(URL.build(scheme=scheme, host=host, port=self._port)) File "/usr/local/lib/python3.9/site-packages/yarl/_url.py", line 237, in build raise TypeError("The port is required to be int.") TypeError: The port is required to be int.

silverdaz commented 8 months ago

@claudiologiudice : it makes more sense to try to install the branch "v2" that we will merge into master soon enough. The version you are installing is a very old one, and we don't have the resources to help you out with that one (We will deprecate it in the coming weeks anyway: ping @sdelatorrep ;-) )