ait-aecid / logdata-anomaly-miner

This tool parses log data and allows to define analysis pipelines for anomaly detection. It was designed to run the analysis with limited resources and lowest possible permissions to make it suitable for production server use.
GNU General Public License v3.0
79 stars 24 forks source link

Debian packages are broken #1329

Open whotwagner opened 5 months ago

whotwagner commented 5 months ago

When aminer was installed via apt on ubuntu 22.04 the folllowing error is thrown:

$ sudo aminer -C -c /etc/aminer/conf.yml
Failed to load configuration from /usr/lib/logdata-anomaly-miner/aminer/YamlConfig.py
Failed to load configuration from /usr/lib/logdata-anomaly-miner/aminer/YamlConfig.py
(<class 'ModuleNotFoundError'>, ModuleNotFoundError("No module named 'yaml'"), <traceback object at 0x7fe2c35ec300>)
Traceback (most recent call last):
  File "/usr/lib/logdata-anomaly-miner/aminer/AminerConfig.py", line 79, in load_config
    aminer_config.load_yaml(yaml_config)
  File "/usr/lib/logdata-anomaly-miner/aminer/YamlConfig.py", line 37, in load_yaml
    import yaml
ModuleNotFoundError: No module named 'yaml'
whotwagner commented 4 months ago

Also on Debian Bullseye:

root@d8acfc109feb:/# aminer -v
Traceback (most recent call last):
  File "/usr/bin/aminer", line 42, in <module>
    from aminer.util.StringUtil import colflame, flame, supports_color  # skipcq: FLK-E402
  File "/usr/lib/logdata-anomaly-miner/aminer/util/__init__.py", line 1, in <module>
    from aminer.util.History import ObjectHistory, LogarithmicBackoffHistory, VolatileLogarithmicBackoffAtomHistory, get_log_int
  File "/usr/lib/logdata-anomaly-miner/aminer/util/History.py", line 16, in <module>
    from aminer.input import AtomHandlerInterface
  File "/usr/lib/logdata-anomaly-miner/aminer/input/__init__.py", line 2, in <module>
    from aminer.input.ByteStreamLineAtomizer import ByteStreamLineAtomizer
  File "/usr/lib/logdata-anomaly-miner/aminer/input/ByteStreamLineAtomizer.py", line 19, in <module>
    from aminer.parsing import MatchContext
  File "/usr/lib/logdata-anomaly-miner/aminer/parsing/__init__.py", line 4, in <module>
    from aminer.parsing.DateTimeModelElement import DateTimeModelElement
  File "/usr/lib/logdata-anomaly-miner/aminer/parsing/DateTimeModelElement.py", line 20, in <module>
    from dateutil.parser import parse
ModuleNotFoundError: No module named 'dateutil'
whotwagner commented 4 months ago

it looks like it was intentional:

root@d8acfc109feb:/# ls -l /etc/aminer/conf-enabled/
total 4
-rw-r--r-- 1 root root 458 Jan 29  2021 Readme.txt
root@d8acfc109feb:/# cat /etc/aminer/conf-enabled/Readme.txt
This directory contains files enabled to be included in the analysis
pipeline configuration. The files are made available by including
this directory within the site packages.

If you have objections enabling all the python site packages stored
on this host within a process running with elevated privileges,
you can also include only some site package components by placing
symlinks here, e.g.

ln -s /usr/lib/python3.6/dist-packages/pytz conf-enabled/pytz
ernstleierzopf commented 3 months ago

Not sure if this is related, but @whotwagner please review the pull requests in the aminer-ansible project. I don't know how the apt installation works, but if it uses the ansible setup, this might fix the issue.

whotwagner commented 3 months ago

Not sure if this is related, but @whotwagner please review the pull requests in the aminer-ansible project. I don't know how the apt installation works, but if it uses the ansible setup, this might fix the issue.

It's not related. The debian packages don't use ansible.