search

StamusNetworks / SELKS

A Suricata based IDS/IPS/NSM distro
https://www.stamus-networks.com/open-source/#selks
GNU General Public License v3.0
1.44k stars 284 forks source link

Default deployment of SELKS dockerized version over Ubuntu 22.04.4 LTS does not work 🐞🐋 #460

Open bleblux opened 5 months ago

bleblux commented 5 months ago

Is there an existing issue for this?

Current Behavior

Default deployment of SELKS dockerized version over Ubuntu 22.04.4 LTS does not work

Expected Behavior

No response

Steps To Reproduce

After execution of , on sudo -E docker compose up -d, I get an error : â Container scirius Error â ´ Container suricata Created
â ´ Container logstash Created
dependency failed to start: container scirius is unhealthy

sudo docker ps -a get a: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1a3f426fd759 elastic/logstash:7.16.1 "/usr/local/bin/dockâ¦" 16 minutes ago Created logstash 970fa5a30ed0 jasonish/suricata:master-amd64 "/etc/suricata/new_eâ¦" 16 minutes ago Created suricata 2943b4580697 elastic/elasticsearch:7.16.1 "/bin/tini -- /usr/lâ¦" 17 minutes ago Up 16 minutes (healthy) 9200/tcp, 9300/tcp elasticsearch bc8cc80984c0 ghcr.io/stamusnetworks/arkimeviewer:master "/start-arkimeviewerâ¦" 17 minutes ago Up 16 minutes 8005/tcp arkime 766b7f98926c ghcr.io/stamusnetworks/scirius:selks "/opt/scirius/bin/stâ¦" 17 minutes ago Up 16 minutes (healthy) 8000/tcp scirius b89a2b76c2de elastic/kibana:7.16.1 "/bin/tini -- /usr/lâ¦" 17 minutes ago Up 16 minutes (healthy) 5601/tcp kibana d9573190b2f3 nginx "/docker-entrypoint.â¦" 17 minutes ago Up 16 minutes 80/tcp, 0.0.0.0:443->443/tcp nginx 55696001a07e jasonish/evebox:master "/docker-entrypoint.â¦" 17 minutes ago Up 16 minutes evebox b7b161ad556b docker:latest "dockerd-entrypoint.â¦" 17 minutes ago Up 16 minutes 2375-2376/tcp cron c46313ea7b2b portainer/portainer-ce "/portainer --logo hâ¦" 23 minutes ago Up 23 minutes 8000/tcp, 9000/tcp, 0.0.0.0:9443->9443/tcp portainer

When try to execute a sudo docker-compose stop I get: ERROR: Named volume "${PWD}/containers-data/scirius/logs:/logs:rw" is used in service "scirius" but no declaration was found in the volumes section.

sudo docker volume ls DRIVER VOLUME NAME local 11a6795b06000a4fff8afec79b895237911498eb3cff8fd45c1f0e9bf106a459 local 902c0c82dcb54c6a9290a1aeac7fdb58d65c44a1ec291d642a142adc02983262 local d9602ef034584c6d871a84230ff0d2bd3ae5b72881507a3e2306698b59e44959 local portainer_data local selks_arkime-config local selks_arkime-logs local selks_arkime-pcap local selks_elastic-data local selks_logstash-sincedb local selks_scirius-data local selks_scirius-static local selks_suricata-logrotate local selks_suricata-rules local selks_suricata-run

For sure, there's a problem with ${PWD} in Ubuntu 22.04.4 LTS

Docker version

Docker version 26.0.0, build 2ae903e

Docker version

docker-compose version 1.29.2, build 5becea4c

OS Version

Ubuntu 22.04.4 LTS

Content of the environnement File

COMPOSE_PROJECT_NAME=selks INTERFACES= -i br0 RESTART_MODE=on-failure SCIRIUS_SECRET_KEY=I3FjKiw4ZCOGq6LTsOdNT0FI5RQ9YeaJ9Azawr5eWKE PWD=${PWD}

Version of SELKS

commit 2fc53910bead2d6057f20d82b62826ba48234097 (HEAD -> master, origin/master, origin/HEAD) Merge: a030b9a 16fc908 Author: Eric Leblond eleblond@stamus-networks.com Date: Mon Sep 11 08:35:37 2023 +0000

Merge branch 'Arkime-fix-v1' into 'master'

Add oui file for Arkime

See merge request devel/SELKS!5

Anything else?

No response

bleblux commented 5 months ago

Replacing "$PWD" in the .env file for "." and in the docker-compose.yml makes the solution start working, all connected EXCEPT moloch that throws an error : {"success":false,"text":"User not found"}

bleblux commented 5 months ago

sudo tail /var/lib/docker/volumes/selks_arkime-logs/_data/viewer.log

WARNING - No users are defined, use node viewer/addUser.js to add one, or turn off auth by unsetting passwordSecret
SECURITY WARNING - when userNameHeader is set, viewHost should be localhost or use iptables
Express server listening on port 8005 in development mode
Tue, 09 Apr 2024 13:42:00 GMT - GET /sessions?expression=ip+%3D%3D+192.168.1.2+%26%26+port+%3D%3D+36058+%26%26+ip+%3D%3D+192.168.1.1+%26%26+port+%3D%3D+53+%26%26+protocols+%3D%3D+udp&date=24 200 41 bytes 20.399 ms

sudo tail /var/lib/docker/volumes/selks_arkime-logs/_data/capture.log

Apr  9 13:42:03 http.c:384 moloch_http_curlm_check_multi_info(): 2/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/bc8cc80984c0 812/158 0ms 51ms
Apr  9 13:42:03 http.c:384 moloch_http_curlm_check_multi_info(): 1/3 ASYNC 201 http://elasticsearch:9200/arkime_dstats/_doc/bc8cc80984c0-1224-5 812/161 0ms 51ms
Apr  9 13:42:05 http.c:384 moloch_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/bc8cc80984c0 812/158 0ms 51ms
Apr  9 13:42:07 http.c:384 moloch_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/bc8cc80984c0 812/158 0ms 51ms
Apr  9 13:42:08 http.c:384 moloch_http_curlm_check_multi_info(): 1/3 ASYNC 201 http://elasticsearch:9200/arkime_dstats/_doc/bc8cc80984c0-1225-5 812/160 0ms 50ms
Apr  9 13:42:09 http.c:384 moloch_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/bc8cc80984c0 812/158 0ms 52ms
Apr  9 13:42:11 http.c:384 moloch_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/bc8cc80984c0 812/158 0ms 51ms
Apr  9 13:42:13 http.c:384 moloch_http_curlm_check_multi_info(): 2/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/bc8cc80984c0 812/158 0ms 51ms
Apr  9 13:42:13 http.c:384 moloch_http_curlm_check_multi_info(): 1/3 ASYNC 201 http://elasticsearch:9200/arkime_dstats/_doc/bc8cc80984c0-1226-5 812/161 0ms 50ms
Apr  9 13:42:15 http.c:384 moloch_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/bc8cc80984c0 812/158 0ms 51ms
bleblux commented 5 months ago

From https://www.howtoforge.com/how-to-install-arkime-moloch-packet-capture-tool-on-ubuntu-22-04/ Tryied to /opt/arkime/db/db.pl http://localhost:9200 init and /opt/arkime/bin/arkime_add_user.sh admin "Moloch SuperAdmin" password --admin /opt/arkime/bin/arkime_add_user.sh selks-user WITHOUT SUCCESS

pevma commented 5 months ago

@bleblux - just confirming as per your chat message. The setup is working fine on previous LTS but not on LTS 22.04.4, correct ?

bleblux commented 5 months ago

Yes!

bleblux commented 5 months ago

sudo docker exec -it arkime sh /opt/arkime/db/db.pl http://elasticsearch:9200/ init /opt/arkime/bin/arkime_add_user.sh selks-user "SELKS Admin User" selks-user --admin /opt/arkime/bin/arkime_add_user.sh moloch moloch moloch --admin --webauth echo 3.2.1 > /etc/.initialized

manually inside the docker gives me access to moloch from web, but It isn't correcly initialized, as if I follow a FPC from the ALERTS dashboard throws an error of inesistent field, understanding that the dialog between elastic and moloch was'nt correctly initialized.