Missing default configuration for reporters

[thorgrin]

I'm on v0.3.2 and I'm trying the Nemea Reporters Configuration module. I'm getting the following error message in the GUI

File /Users/petrstehlik/dev/liberouter-gui/modules/Nemea-Dashboard/api/modules/nemea/config.yaml not found 

And this one in the apache log:

Fri Aug 04 12:17:48.943865 2017] [wsgi:error] [pid 9602] [remote 10.0.2.2:42762] Caught error!
[Fri Aug 04 12:17:48.943921 2017] [wsgi:error] [pid 9602] [remote 10.0.2.2:42762] {'status_code': 404, 'error': True, 'message': 'File /Users/petrstehlik/dev/liberouter-gui/modules/Nemea-Dashboard/api/modules/nemea/config.yaml not found'}

I guess that there must be a way to configure the path to the reporter configuration, but I cannot find it. Maybe the error message could be made more helpful by pointing out the correct configuration option that needs to be set?

[petrstehlik]

The problem is in Nemea-Dashboard module was a separate config file. This config file is located at the Nemea-Dashboard module (link: https://github.com/CESNET/Nemea-Dashboard/blob/liberouter-gui/api/modules/nemea/config.ini).

I also added a smarter way on using the config files. If nemea section is present in the main config file (must contain database, collection, reporter_config items) the local config is ignored.

The path is already changed to be relative and example config can be found here: https://github.com/CESNET/Nemea-Framework/blob/master/pycommon/reporter_config/example.yaml

[thorgrin]

I've tried adding

[nemea]
database = nemeadb
collection = alerts_new
reporters_config = /etc/nemea/reporters-config.yaml

to /var/www/html/liberouter-gui/api/config.ini, but now I get following error:

[Fri Aug 04 15:26:10.261101 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563] mod_wsgi (pid=9667): Target WSGI script '/var/www/html/liberouter-gui/api/wsgi.py' cannot be loaded as Python module.
[Fri Aug 04 15:26:10.261156 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563] mod_wsgi (pid=9667): Exception occurred processing WSGI script '/var/www/html/liberouter-gui/api/wsgi.py'.
[Fri Aug 04 15:26:10.261186 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563] Traceback (most recent call last):
[Fri Aug 04 15:26:10.261238 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563]   File "/var/www/html/liberouter-gui/api/wsgi.py", line 12, in <module>
[Fri Aug 04 15:26:10.261243 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563]     from liberouterapi import app as application
[Fri Aug 04 15:26:10.261267 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563]   File "/var/www/html/liberouter-gui/api/liberouterapi/__init__.py", line 77, in <module>
[Fri Aug 04 15:26:10.261280 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563]     import_modules()
[Fri Aug 04 15:26:10.261304 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563]   File "/var/www/html/liberouter-gui/api/liberouterapi/bootstrap.py", line 37, in import_modules
[Fri Aug 04 15:26:10.261308 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563]     fromlist=[mod_name])
[Fri Aug 04 15:26:10.261332 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563]   File "/var/www/html/liberouter-gui/api/liberouterapi/modules/nemea/__init__.py", line 9, in <module>
[Fri Aug 04 15:26:10.261335 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563]     nemea_db = db.socket[config.modules['nemea']['database']]
[Fri Aug 04 15:26:10.261355 2017] [wsgi:error] [pid 9667] [remote 10.0.2.2:49563] KeyError: 'nemea'

I'm running the v0.3.2 frontent release with git at commit a53814a7dc398ea7f1fa5af15b16bbef8815fdab

[thorgrin]

So I've tried to put correct path to a very simple configuration to the configuration of the nemea dashboard and now it runs and finds the file. The reporter configuration looks like this:

custom_actions:
  - id: store_mongo
    mongo:
       host: localhost
       db: nemeadb
       collection: alerts_new
  - id: store_warden
    warden:
      url: https://localhost/warden/
  - id: file
    file:
       path: /dev/stdout
rules:
- id: 1
  condition: Null
  actions:
  - store_mongo
  - store_warden

The problem is that the Nemea Reporters Configuration is empty and I cannot do anything with it (the controls do work). I'm still running the v0.3.2 frontent release with git at commit a53814a

[petrstehlik]

Sorry for my mistakes, both should be fixed with latest commit to the NEMEA Dashboard submodule here: CESNET/Nemea-Dashboard@2a90fc4

[thorgrin]

I can confirm, that the reporter configuration loaded and showed the stored configuration. However, the first problem does not seem to be fixed:

/var/log/httpd/ssl_error_log

[Tue Aug 08 09:21:48.303490 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672] mod_wsgi (pid=19074): Target WSGI script '/var/www/html/liberouter-gui/api/wsgi.py' cannot be loaded as Python module.
[Tue Aug 08 09:21:48.303541 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672] mod_wsgi (pid=19074): Exception occurred processing WSGI script '/var/www/html/liberouter-gui/api/wsgi.py'.
[Tue Aug 08 09:21:48.303567 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672] Traceback (most recent call last):
[Tue Aug 08 09:21:48.303607 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]   File "/var/www/html/liberouter-gui/api/wsgi.py", line 12, in <module>
[Tue Aug 08 09:21:48.303611 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]     from liberouterapi import app as application
[Tue Aug 08 09:21:48.303630 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]   File "/var/www/html/liberouter-gui/api/liberouterapi/__init__.py", line 77, in <module>
[Tue Aug 08 09:21:48.303633 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]     import_modules()
[Tue Aug 08 09:21:48.303649 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]   File "/var/www/html/liberouter-gui/api/liberouterapi/bootstrap.py", line 37, in import_modules
[Tue Aug 08 09:21:48.303652 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]     fromlist=[mod_name])
[Tue Aug 08 09:21:48.303669 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]   File "/var/www/html/liberouter-gui/api/liberouterapi/modules/nemea/__init__.py", line 21, in <module>
[Tue Aug 08 09:21:48.303672 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]     from .Reporters import *
[Tue Aug 08 09:21:48.303709 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]   File "/var/www/html/liberouter-gui/api/liberouterapi/modules/nemea/Reporters.py", line 20, in <module>
[Tue Aug 08 09:21:48.303711 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672]     if 'reporters_config' not in config.modules['nemea']:
[Tue Aug 08 09:21:48.303727 2017] [wsgi:error] [pid 19074] [remote 10.0.2.2:39672] KeyError: 'nemea'

/var/www/html/liberouter-gui/api/liberouterapi/modules/nemea/config.ini

[nemea]
database = nemeadbaa
collection = alerts_newaa
reporters_config = /etc/nemea/reporters-config.ymlaa

/var/www/html/liberouter-gui/api/config.ini

some more default stuff
...
[nemea]
database = nemeadb
collection = alerts_new
reporters_config = /etc/nemea/reporters-config.yaml

[petrstehlik]

The configurator needs a couple of fixes for this, I didn't realize that in the first place.

[petrstehlik]

Back-end part of NEMEA Reporter Configurator now works as expected.