[Bug] Malware Analyzer docker exits with code 3

mostwanted002 commented 1 year ago

What happened

When starting up the malware analyzer docker using python3 start.py --malware_tools_analyzer, the docker-compose logs show that the container doesn't start successfully and exits with code 3. This leads to failure in file analysis if any analyzers from malware tools are selected.

Environment

OS: Linux (Arch 6.0.12-arch1-1)
IntelOwl version: 4.1.2
Python: 3.10.8
Docker: 20.10.21
Docker-Compose: 2.14.0

What did you expect to happen

The expected behavior is for the container to start normally and stay up until stopped.

How to reproduce your issue

Follow the installation instruction mentioned in the documentation
Start the IntelOwl using python3 start.py --malware_tools_analyzer

Error messages and logs

docker-compose output with status code 3

mostwanted002 commented 1 year ago

On further investigation into the behavior, when I tried to build and run the malware_tools_analyzer docker standalone, the issue became more clear.

The error says:

Clamd was NOT notified: Can't connect to clamd through /var/run/clamav/clamd.ctl: Permission denied

This was fixed by changing the permissions of the folder from 750 to 777 in Dockerfile, which is just a quick and dirty workaround as it is an insecure way of making the clamd socket world writable.

When that issue was fixed, another issue was detected, this time a python dependency error.

[2022-12-22 02:32:10 +0000] [17] [INFO] Starting gunicorn 20.1.0
[2022-12-22 02:32:10 +0000] [17] [DEBUG] Arbiter booted
[2022-12-22 02:32:10 +0000] [17] [INFO] Listening at: http://0.0.0.0:4002/ (17)
[2022-12-22 02:32:10 +0000] [17] [INFO] Using worker: sync
[2022-12-22 02:32:10 +0000] [24] [INFO] Booting worker with pid: 24
[2022-12-22 02:32:10 +0000] [17] [DEBUG] 1 workers
[2022-12-22 02:32:10 +0000] [24] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/deploy/malware_tools_analyzers-flask/app.py", line 13, in <module>
    from flask import Flask, safe_join
ImportError: cannot import name 'safe_join' from 'flask' (/usr/local/lib/python3.8/site-packages/flask/__init__.py)
[2022-12-22 02:32:10 +0000] [24] [INFO] Worker exiting (pid: 24)
[2022-12-22 02:32:10 +0000] [17] [INFO] Shutting down: Master
[2022-12-22 02:32:10 +0000] [17] [INFO] Reason: Worker failed to boot.

mlodic commented 1 year ago

hey thank you for your bug report.

the python dependency should be fixed here: https://github.com/intelowlproject/IntelOwl/commit/7889c68e81f3c32617d6589dc616a5f6bc4fcd4b. That was caused by an update of the Flask dependency.

I am about to create a new release with that bug fix. Please follow the project new release for a direct update.

On the contrary, the file permission error is a more sneaky bug that is not always replicable. I changed the permissions in the Dockerfile for /var/run/clamav from 750 to 755 cause I think that the updater is running with a different user from the one which is running the application so both of them need access to that folder. This should fix the problem.

mlodic commented 1 year ago

should be fixed with https://github.com/intelowlproject/IntelOwl/commit/673b6c57dced7697bc82b29d31d230ee021f5979

intelowlproject / IntelOwl