search

nsacyber / WALKOFF

A flexible, easy to use, automation framework allowing users to integrate their capabilities and devices to cut through the repetitive, tedious tasks slowing them down. #nsacyber
https://nsacyber.github.io/WALKOFF/
Other
1.2k stars 222 forks source link

Development branch isn't listing walkoff apps #245

Open peterclemenko opened 5 years ago

peterclemenko commented 5 years ago

On the development branch, walkoff isn't listing the walkoff apps in the UI or in the editor.

peterclemenko commented 4 years ago

I have a theory that what might be causing this on my end is that umpire might be having a hard time pushing the containers for walkoff apps to docker. Similar to what was happening in #240

peterclemenko commented 4 years ago

So I was able to get an image list and this is what showed up:

docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE walkoff_bootloader latest 7660c1f0526e About a minute ago 427MB

0e0d98c4a130 About a minute ago 415MB bitnami/nginx 51758d19eea8 About an hour ago 77.9MB 127.0.0.1:5000/api_gateway a944e24a7201 8 days ago 268MB python 3.7.4-slim-buster a8c0694fba17 12 days ago 179MB postgres f88dfa384cc4 12 days ago 348MB redis de25a81a5a0b 12 days ago 98.2MB minio/minio 8869bca0366f 2 weeks ago 51MB portainer/portainer 4cda95efb0e4 2 weeks ago 80.6MB registry f32a97de94e1 7 months ago 25.8MB from the looks of it, umpire and the apps and the app sdk aren't pushing on walkoff.ps1 up
adpham95 commented 4 years ago

Is this on master or development?

peterclemenko commented 4 years ago

This is on development

peterclemenko commented 4 years ago

So on master I just had an error happen when I did the hack to get aiodocker to work

Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/aiohttp/client.py", line 380, in _request url = URL(str_or_url) File "/usr/local/lib/python3.7/site-packages/yarl/init.py", line 160, in new raise ValueError("Invalid URL: host is required for abolute urls.") ValueError: Invalid URL: host is required for abolute urls.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/tenacity/_asyncio.py", line 46, in call result = yield from fn(*args, **kwargs) File "/c/users/th3fl/desktop/walkoff/bootloader/bootloader.py", line 342, in wait_for_registry raise e File "/c/users/th3fl/desktop/walkoff/bootloader/bootloader.py", line 335, in wait_for_registry async with self.session.get(f"http://{DOCKER_HOST_IP}:5000") as resp: File "/usr/local/lib/python3.7/site-packages/aiohttp/client.py", line 1012, in aenter self._resp = await self._coro File "/usr/local/lib/python3.7/site-packages/aiohttp/client.py", line 382, in _request raise InvalidURL(str_or_url) aiohttp.client_exceptions.InvalidURL: http://:5000

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/c/users/th3fl/desktop/walkoff/bootloader/bootloader.py", line 543, in asyncio.run(Bootloader.run()) File "/usr/local/lib/python3.7/asyncio/runners.py", line 43, in run return loop.run_until_complete(main) File "/usr/local/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete return future.result() File "/c/users/th3fl/desktop/walkoff/bootloader/bootloader.py", line 326, in run await getattr(bootloader, args.command)() File "/c/users/th3fl/desktop/walkoff/bootloader/bootloader.py", line 401, in up await self.wait_for_registry() File "/usr/local/lib/python3.7/site-packages/tenacity/_asyncio.py", line 43, in call do = self.iter(retry_state=retry_state) File "/usr/local/lib/python3.7/site-packages/tenacity/init.py", line 333, in iter six.raise_from(retry_exc, fut.exception()) File "", line 3, in raise_from tenacity.RetryError: RetryError[<Future at 0x7feabc74c610 state=finished raised InvalidURL>]

maybe this is related?

peterclemenko commented 4 years ago

So I manually built umpire and got these errors:

ERROR: docker-compose 1.24.1 has requirement docker[ssh]<4.0,>=3.7.0, but you'll have docker 4.1.0 which is incompatible. ERROR: docker-compose 1.24.1 has requirement PyYAML<4.3,>=3.10, but you'll have pyyaml 5.1.2 which is incompatible. ERROR: docker-compose 1.24.1 has requirement requests!=2.11.0,!=2.12.2,!=2.18.0,<2.21,>=2.6.1, but you'll have requests 2.22.0 which is incompatible. ERROR: fastapi 0.42.0 has requirement pydantic<=0.32.2,>=0.32.2, but you'll have pydantic 1.0 which is incompatible.

peterclemenko commented 4 years ago

Seems to be related to these: https://github.com/docker/compose/issues/6617 https://github.com/Azure/iotedgehubdev/issues/190

peterclemenko commented 4 years ago

Maybe this will fix it, I don't know https://github.com/docker/compose/releases/tag/1.25.0-rc4

peterclemenko commented 4 years ago

so it seems that umpire isn't building or pushing, doing that manually seems to fix this

docker build -f umpire/Dockerfile -t 127.0.0.1:5000/umpire . docker push 127.0.0.1:5000/umpire

this seems to be similar to the problem I'm having with nginx and the api gateway