search

quarkslab / irma

IRMA is an asynchronous & customizable analysis system for suspicious files.
https://irma.quarkslab.com
Apache License 2.0
269 stars 55 forks source link

Unable to run IRMA without Vagrant #76

Closed marcosepp closed 3 years ago

marcosepp commented 3 years ago

I tried to install IRMA in production mode to pre-configured Debian10 hosts which were deployed using Debian OpenStack cloud images.

Ansible playbooks were successful after some modifications to the code, but application is not working properly. I checked system logs and found that brain and frontend services are not started. Thus frontend is serving the UI but there aren't any probes registred. It might be because I've made some serious code changes and that's why it's not working. Maybe you can guide me, what might be causing the errors and how can I fix it?

Brain journalctl error:

Feb 13 16:53:19 irma-brain systemd[1]: Started irma.result_app.service.
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]: Traceback (most recent call last):
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     "__main__", mod_spec)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     exec(code, run_globals)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/releases/20210213163311/brain/results_tasks.py", line 81, in <module>
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     results_app.worker_main(options)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/app/base.py", line 206, in worker_main
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     app=self).execute_from_commandline(argv)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 311, in execute_from_commandline
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     return self.handle_argv(self.prog_name, argv[1:])
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 377, in handle_argv
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     return self(*args, **options)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 274, in __call__
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     ret = self.run(*args, **kwargs)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/bin/worker.py", line 194, in run
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     pool_cls = (concurrency.get_implementation(pool_cls) or
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/concurrency/__init__.py", line 29, in get_implementation
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     return symbol_by_name(cls, ALIASES)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/kombu/utils/__init__.py", line 96, in symbol_by_name
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     module = imp(module_name, package=package, **kwargs)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     return _bootstrap._gcd_import(name[level:], package, level)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/concurrency/prefork.py", line 20, in <module>
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     from celery.concurrency.base import BasePool
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/concurrency/base.py", line 21, in <module>
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     from celery.utils import timer2
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/utils/timer2.py", line 19
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     from kombu.async.timer import Entry, Timer as Schedule, to_timestamp, logger
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:                    ^
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]: SyntaxError: invalid syntax
Feb 13 16:53:20 irma-brain systemd[1]: irma.result_app.service: Main process exited, code=exited, status=1/FAILURE
Feb 13 16:53:20 irma-brain systemd[1]: irma.result_app.service: Failed with result 'exit-code'.

Frontend journalctl logs:

Feb 13 16:32:36 irma-frontend systemd[1]: Started irma.frontend_app.service.
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]: Traceback (most recent call last):
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     "__main__", mod_spec)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     exec(code, run_globals)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/releases/20210213163208/api/tasks/frontend_app.py", line 175, in <module>
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     frontend_app.worker_main(options)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/app/base.py", line 206, in worker_main
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     app=self).execute_from_commandline(argv)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 311, in execute_from_commandline
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     return self.handle_argv(self.prog_name, argv[1:])
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 377, in handle_argv
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     return self(*args, **options)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 274, in __call__
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     ret = self.run(*args, **kwargs)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/bin/worker.py", line 194, in run
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     pool_cls = (concurrency.get_implementation(pool_cls) or
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/concurrency/__init__.py", line 29, in get_implementation
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     return symbol_by_name(cls, ALIASES)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/kombu/utils/__init__.py", line 96, in symbol_by_name
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     module = imp(module_name, package=package, **kwargs)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     return _bootstrap._gcd_import(name[level:], package, level)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/concurrency/prefork.py", line 20, in <module>
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     from celery.concurrency.base import BasePool
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/concurrency/base.py", line 21, in <module>
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     from celery.utils import timer2
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/utils/timer2.py", line 19
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     from kombu.async.timer import Entry, Timer as Schedule, to_timestamp, logger
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:                    ^
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]: SyntaxError: invalid syntax
Feb 13 16:32:37 irma-frontend systemd[1]: irma.frontend_app.service: Main process exited, code=exited, status=1/FAILURE
Feb 13 16:32:37 irma-frontend systemd[1]: irma.frontend_app.service: Failed with result 'exit-code'.

Marco

ch0k0bn commented 3 years ago

Irma v2 is tested mainly on Debian9 (python3 version is 3.5 instead of 3.7). Do you have a Debian9 base image available?

marcosepp commented 3 years ago

I will try Debian9.

Thanks for your reply!

marcosepp commented 3 years ago

I tried Ubuntu 18.04 because of my environment specifics and it works fine now. I have to test it with Debian 9 as well but this is irrelevant from this issue.