Open Mauro2k opened 5 years ago
What is the output of selks-healthcheck_stamus
?
Output selks_healthcheck_stamus:
selks-user@SELKS:~$ selks-health-check_stamus ● suricata.service - LSB: Next Generation IDS/IPS Loaded: loaded (/etc/init.d/suricata; generated; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:59 -03; 49min ago Docs: man:systemd-sysv-generator(8) Process: 661 ExecStart=/etc/init.d/suricata start (code=exited, status=0/SUCCESS) Tasks: 10 (limit: 4915) CGroup: /system.slice/suricata.service └─873 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v --user=logstash ● elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:59 -03; 49min ago Docs: http://www.elastic.co Main PID: 666 (java) Tasks: 82 (limit: 4915) CGroup: /system.slice/elasticsearch.service ├─ 666 /usr/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOcc…et └─1090 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:57 -03; 49min ago Main PID: 415 (java) Tasks: 36 (limit: 4915) CGroup: /system.slice/logstash.service └─415 /usr/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccu…sh ● kibana.service - Kibana Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:58 -03; 49min ago Main PID: 487 (node) Tasks: 11 (limit: 4915) CGroup: /system.slice/kibana.service └─487 /usr/share/kibana/bin/../node/bin/node --no-warnings --max-http-header-size=65536 /usr/share/kibana/bin/../src/cli …ml ● evebox.service - EveBox Server Loaded: loaded (/lib/systemd/system/evebox.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:58 -03; 49min ago Main PID: 483 (evebox) Tasks: 13 (limit: 4915) CGroup: /system.slice/evebox.service └─483 /usr/bin/evebox server ● molochviewer-selks.service - Moloch Viewer Loaded: loaded (/etc/systemd/system/molochviewer-selks.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Fri 2019-05-24 15:10:50 -03; 42min ago Process: 1432 ExecStart=/bin/sh -c /data/moloch/bin/node viewer.js -c /data/moloch/etc/config.ini >> /data/moloch/logs/viewer.log 2>&1 (code=exited, status=1/FAILURE) Main PID: 1432 (code=exited, status=1/FAILURE) ● molochpcapread-selks.service - Moloch Pcap Read Loaded: loaded (/etc/systemd/system/molochpcapread-selks.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:06:42 -03; 46min ago Main PID: 1296 (sh) Tasks: 5 (limit: 4915) CGroup: /system.slice/molochpcapread-selks.service ├─1296 /bin/sh -c /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini -m -s -R /data/nsm/ >> /data/moloch/log…&1 └─1297 /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini -m -s -R /data/nsm/ error: <class 'socket.error'>, [Errno 13] Permission denied: file: /usr/lib/python2.7/socket.py line: 228 ii elasticsearch 6.8.0 all Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html ii elasticsearch-curator 5.7.6 amd64 Have indices in Elasticsearch? This is the tool for you!\n\nLike a museum curator manages the exhibits and collections on display, \nElasticsearch Curator helps you curate, or manage your indices. ii evebox 1:0.10.2 amd64 no description given ii kibana 6.8.0 amd64 Explore and visualize your Elasticsearch data ii kibana-dashboards-stamus 2019030501 amd64 Kibana 6 dashboard templates. ii logstash 1:6.8.0-1 all An extensible logging pipeline ii moloch 1.8.0-1 amd64 Moloch Full Packet System ii scirius 3.2.0-1 amd64 Django application to manage Suricata ruleset ii suricata 2019040702-0stamus0 amd64 Suricata open source multi-thread IDS/IPS/NSM system. Filesystem Type Size Used Avail Use% Mounted on udev devtmpfs 4.9G 0 4.9G 0% /dev tmpfs tmpfs 1001M 17M 984M 2% /run /dev/sda1 ext4 334G 6.9G 310G 3% / tmpfs tmpfs 4.9G 0 4.9G 0% /dev/shm tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs tmpfs 4.9G 0 4.9G 0% /sys/fs/cgroup tmpfs tmpfs 1001M 4.0K 1001M 1% /run/user/112 tmpfs tmpfs 1001M 0 1001M 0% /run/user/1000
Ok thanks for the feedback. Are there events on the “SN ALL” dashboard ? And are there alerts in “SN ALERTS” dashboard ?
-- Regards, Peter Manev
On 24 May 2019, at 20:54, Mac notifications@github.com wrote:
Output selks_healthcheck_stamus:
selks-user@SELKS:~$ selks-health-check_stamus ● suricata.service - LSB: Next Generation IDS/IPS Loaded: loaded (/etc/init.d/suricata; generated; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:59 -03; 49min ago Docs: man:systemd-sysv-generator(8) Process: 661 ExecStart=/etc/init.d/suricata start (code=exited, status=0/SUCCESS) Tasks: 10 (limit: 4915) CGroup: /system.slice/suricata.service └─873 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v --user=logstash ● elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:59 -03; 49min ago Docs: http://www.elastic.co Main PID: 666 (java) Tasks: 82 (limit: 4915) CGroup: /system.slice/elasticsearch.service ├─ 666 /usr/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOcc…et └─1090 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:57 -03; 49min ago Main PID: 415 (java) Tasks: 36 (limit: 4915) CGroup: /system.slice/logstash.service └─415 /usr/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccu…sh ● kibana.service - Kibana Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:58 -03; 49min ago Main PID: 487 (node) Tasks: 11 (limit: 4915) CGroup: /system.slice/kibana.service └─487 /usr/share/kibana/bin/../node/bin/node --no-warnings --max-http-header-size=65536 /usr/share/kibana/bin/../src/cli …ml ● evebox.service - EveBox Server Loaded: loaded (/lib/systemd/system/evebox.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:03:58 -03; 49min ago Main PID: 483 (evebox) Tasks: 13 (limit: 4915) CGroup: /system.slice/evebox.service └─483 /usr/bin/evebox server ● molochviewer-selks.service - Moloch Viewer Loaded: loaded (/etc/systemd/system/molochviewer-selks.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Fri 2019-05-24 15:10:50 -03; 42min ago Process: 1432 ExecStart=/bin/sh -c /data/moloch/bin/node viewer.js -c /data/moloch/etc/config.ini >> /data/moloch/logs/viewer.log 2>&1 (code=exited, status=1/FAILURE) Main PID: 1432 (code=exited, status=1/FAILURE) ● molochpcapread-selks.service - Moloch Pcap Read Loaded: loaded (/etc/systemd/system/molochpcapread-selks.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-05-24 15:06:42 -03; 46min ago Main PID: 1296 (sh) Tasks: 5 (limit: 4915) CGroup: /system.slice/molochpcapread-selks.service ├─1296 /bin/sh -c /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini -m -s -R /data/nsm/ >> /data/moloch/log…&1 └─1297 /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini -m -s -R /data/nsm/ error: <class 'socket.error'>, [Errno 13] Permission denied: file: /usr/lib/python2.7/socket.py line: 228 ii elasticsearch 6.8.0 all Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html ii elasticsearch-curator 5.7.6 amd64 Have indices in Elasticsearch? This is the tool for you!\n\nLike a museum curator manages the exhibits and collections on display, \nElasticsearch Curator helps you curate, or manage your indices. ii evebox 1:0.10.2 amd64 no description given ii kibana 6.8.0 amd64 Explore and visualize your Elasticsearch data ii kibana-dashboards-stamus 2019030501 amd64 Kibana 6 dashboard templates. ii logstash 1:6.8.0-1 all An extensible logging pipeline ii moloch 1.8.0-1 amd64 Moloch Full Packet System ii scirius 3.2.0-1 amd64 Django application to manage Suricata ruleset ii suricata 2019040702-0stamus0 amd64 Suricata open source multi-thread IDS/IPS/NSM system. Filesystem Type Size Used Avail Use% Mounted on udev devtmpfs 4.9G 0 4.9G 0% /dev tmpfs tmpfs 1001M 17M 984M 2% /run /dev/sda1 ext4 334G 6.9G 310G 3% / tmpfs tmpfs 4.9G 0 4.9G 0% /dev/shm tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs tmpfs 4.9G 0 4.9G 0% /sys/fs/cgroup tmpfs tmpfs 1001M 4.0K 1001M 1% /run/user/112 tmpfs tmpfs 1001M 0 1001M 0% /run/user/1000
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
Only some data in SN-ALL
SN-ALL
SNALERTS
Logstash has full data for what I see in "Discover"
It seems there is traffic just no alerts. Anything that has recently changed on your network ?
No, no change. What strikes me is that I have traffic, but not alerts. I have generated alerts of all kinds and they do not appear. I do not know what is happening :(
In the configuration file "suricata.yaml" I have configured my vLans within HOME_NET:
This is fine, right?
What is the output of tail -50 /var/log/suricata/suricata.log ?
-- Regards, Peter Manev
On 24 May 2019, at 22:19, Mac notifications@github.com wrote:
No, no change. What strikes me is that I have traffic, but not alerts. I have generated alerts of all kinds and they do not appear. I do not know what is happening :(
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
Output:
selks-user@SELKS:/var/log/suricata$ tail -50 /var/log/suricata/suricata.log
[818] 24/5/2019 -- 17:27:59 - (output-json-stats.c:468)
Is that the end of the file? There should be something more to the bottom i think. The network definition looks ok yes.
Paste again:
selks-user@SELKS:/var/log/suricata$ tail -50 /var/log/suricata/suricata.log
[1042] 26/5/2019 -- 02:01:44 - (detect-engine-loader.c:351)
ok seems better. what is the timestamp output of the logs by the command:
grep '"event_type": "alert"' /var/log/suricata/eve-alert.json | tail -1
No data..
The command dies not appear to be the same as I quoted in my previous response - seems there is a space missing in your execution ?
-- Regards, Peter Manev
On 27 May 2019, at 17:04, Mac notifications@github.com wrote:
No data..
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
The file eve-alert.json does not exist. There is the file eve.json. Is the same?
Yes - sorry - eve.json
So it means there are no alerts generated - and having in mind Suricata seems to be running there could be something else that changed ?
-- Regards, Peter Manev
On 27 May 2019, at 18:40, Mac notifications@github.com wrote:
The file eve-alert.json does not exist. There is the file eve.json. Is the same?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
I have not changed anything. It is the initial installation. I can not find the problem. I'm disoriented.
If you have not changed anything - it means that maybe something changed on the mirror/traffic port ?
I've thought about that. But if it was a problem in the PortMirroring configuration, why do I see the packages in the EVEBOX?
I remind you that this installation is made on an ESXi.
Do you see Alerts in Evebox where you do not see them for that amount of time in Scirius/Kibana ?
No, I only see traffic but not alerts!
ok - so all apps do not see alerts as there are none in the DB (ES) so in that case the change must be elsewhere, somewhere on the host or something changed in the traffic. But i myself am not sure what exactly could that be. If you do tcpdump do you see something abnormal in the traffic capture (for example via wireshark if the size of the pcap is not that huge)?
I spent some time analyzing the traffic with tcpdump and I do not see anything strange. The sniffing interface receives traffic from all the vlans.
Any offloading or warnings observed by wireshark ?
Do you have vlan tracking enabled in /etc/suricata/selks5-addins.yaml
? ( https://github.com/OISF/suricata/blob/master/suricata.yaml.in#L1229 )
Yes, i Have it.
Flow-timeouts: by default
I just checked the fast.log file and I see that it has records. When reviewing it, I notice that there are some alarms since 25/5 but only appears in the DASHBOARD the last one detected.
Fast.log
Dashboard:
At the same time I have performed the following test and it does not appear:
curl -A "BlackSun" www.google.com
Before the reinstallation I did it.
This is very strange, I do not understand what is happening
The records above appear to be from 25 May and 30 May. (Also just for info SELKS does not utilize fast.log as it is legacy. )
So those same records should be present in /var/log/suricata/eve.json
Could you try this - (i realized there was an extra space befor "alert"
in the command before)
grep '"event_type":"alert"' /var/log/suricata/eve.json | tail -5 | jq .
Also check the timezone set up onn the box /cmd (date
) and in Scirius (Settings
, from the drop down menu in the upper left corner)
selks-user@SELKS:/var/log/suricata$ grep '"event_type":"alert"' /var/log/suricata/eve.json | tail -5 | jq .
{
"timestamp": "2019-05-31T10:14:19.582621-0300",
"flow_id": 119216089588701,
"in_iface": "ens192",
"event_type": "alert",
"src_ip": "192.168.21.132",
"src_port": 57621,
"dest_ip": "192.168.21.255",
"dest_port": 57621,
"proto": "UDP",
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 2027397,
"rev": 1,
"signature": "ET POLICY Spotify P2P Client",
"category": "Not Suspicious Traffic",
"severity": 3,
"metadata": {
"updated_at": [
"2019_05_30"
],
"performance_impact": [
"Low"
],
"created_at": [
"2019_05_30"
],
"signature_severity": [
"Minor"
],
"deployment": [
"Internal"
],
"attack_target": [
"Client_Endpoint"
],
"affected_product": [
"Windows_Client_Apps"
]
}
},
"app_proto": "failed",
"flow": {
"pkts_toserver": 1,
"pkts_toclient": 0,
"bytes_toserver": 86,
"bytes_toclient": 0,
"start": "2019-05-31T10:14:19.582621-0300"
},
"payload": "U3BvdFVkcDBLCKgSAE27EAABAARIlcIDo3coC8mysGtyrPsNoy5gZZI4DZY=",
"payload_printable": "SpotUdp0K....M......H....w(....kr..\r..e.8\r.", "stream": 0, "packet": "////////aPcoseVRCABFAABIuJFAAEAR1T/AqBWEwKgV/+EV4RUANPg3U3BvdFVkcDBLCKgSAE27EAABAARIlcIDo3coC8mysGtyrPsNoy5gZZI4DZY=", "packet_info": { "linktype": 1 } } { "timestamp": "2019-05-31T10:18:10.354490-0300", "flow_id": 2149754235676858, "in_iface": "ens192", "event_type": "alert", "src_ip": "192.168.21.137", "src_port": 57621, "dest_ip": "192.168.21.255", "dest_port": 57621, "proto": "UDP", "alert": { "action": "allowed", "gid": 1, "signature_id": 2027397, "rev": 1, "signature": "ET POLICY Spotify P2P Client", "category": "Not Suspicious Traffic", "severity": 3, "metadata": { "updated_at": [ "2019_05_30" ], "performance_impact": [ "Low" ], "created_at": [ "2019_05_30" ], "signature_severity": [ "Minor" ], "deployment": [ "Internal" ], "attack_target": [ "Client_Endpoint" ], "affected_product": [ "Windows_Client_Apps" ] } }, "app_proto": "failed", "flow": { "pkts_toserver": 1, "pkts_toclient": 0, "bytes_toserver": 86, "bytes_toclient": 0, "start": "2019-05-31T10:18:10.354490-0300" }, "payload": "U3BvdFVkcDD3W4ozF+ltNAABAARIlcIDNICyxlv9NPurvFhNklmUWDhYuGE=", "payload_printable": "SpotUdp0.[.3..m4....H...4...[.4...XM.Y.X8X.a", "stream": 0, "packet": "////////UHudawebCABFAABILpsAAIARXzHAqBWJwKgV/+EV4RUANGMHU3BvdFVkcDD3W4ozF+ltNAABAARIlcIDNICyxlv9NPurvFhNklmUWDhYuGE=", "packet_info": { "linktype": 1 } } { "timestamp": "2019-05-31T10:18:08.840777-0300", "flow_id": 1644968876758089, "in_iface": "ens192", "event_type": "alert", "src_ip": "192.168.21.33", "src_port": 57621, "dest_ip": "192.168.21.255", "dest_port": 57621, "proto": "UDP", "alert": { "action": "allowed", "gid": 1, "signature_id": 2027397, "rev": 1, "signature": "ET POLICY Spotify P2P Client", "category": "Not Suspicious Traffic", "severity": 3, "metadata": { "updated_at": [ "2019_05_30" ], "performance_impact": [ "Low" ], "created_at": [ "2019_05_30" ], "signature_severity": [ "Minor" ], "deployment": [ "Internal" ], "attack_target": [ "Client_Endpoint" ], "affected_product": [ "Windows_Client_Apps" ] } }, "app_proto": "failed", "flow": { "pkts_toserver": 1, "pkts_toclient": 0, "bytes_toserver": 86, "bytes_toclient": 0, "start": "2019-05-31T10:18:08.840777-0300" }, "payload": "U3BvdFVkcDA+sSRJnnVEtgABAARIlcIDgxZH3wMxiDh2y9i/47lonyA9QCM=", "payload_printable": "SpotUdp0>.$I.uD.....H.....G..1.8v.....h. =@#", "stream": 0, "packet": "////////BNOwwWJKCABFAABI0hdAAEARvBzAqBUhwKgV/+EV4RUANGUHU3BvdFVkcDA+sSRJnnVEtgABAARIlcIDgxZH3wMxiDh2y9i/47lonyA9QCM=", "packet_info": { "linktype": 1 } } { "timestamp": "2019-05-31T10:18:16.343879-0300", "flow_id": 476765690019655, "in_iface": "ens192", "event_type": "alert", "src_ip": "192.168.7.116", "src_port": 57621, "dest_ip": "192.168.7.255", "dest_port": 57621, "proto": "UDP", "alert": { "action": "allowed", "gid": 1, "signature_id": 2027397, "rev": 1, "signature": "ET POLICY Spotify P2P Client", "category": "Not Suspicious Traffic", "severity": 3, "metadata": { "updated_at": [ "2019_05_30" ], "performance_impact": [ "Low" ], "created_at": [ "2019_05_30" ], "signature_severity": [ "Minor" ], "deployment": [ "Internal" ], "attack_target": [ "Client_Endpoint" ], "affected_product": [ "Windows_Client_Apps" ] } }, "app_proto": "failed", "flow": { "pkts_toserver": 1, "pkts_toclient": 0, "bytes_toserver": 86, "bytes_toclient": 0, "start": "2019-05-31T10:18:16.343879-0300" }, "payload": "U3BvdFVkcDB80YrwoXSOTQABAARIlcIDwCdMBYKAIrYGahYbFqxWT9BhF5I=", "payload_printable": "SpotUdp0|....t.M....H....'L...\"..j....VO.a..", "stream": 0, "packet": "////////jBZFiQcXCABFAABInahAAEARDDnAqAd0wKgH/+EV4RUANL8iU3BvdFVkcDB80YrwoXSOTQABAARIlcIDwCdMBYKAIrYGahYbFqxWT9BhF5I=", "packet_info": { "linktype": 1 } } { "timestamp": "2019-05-31T10:18:40.125923-0300", "flow_id": 615731505982435, "in_iface": "ens192", "event_type": "alert", "src_ip": "192.168.21.132", "src_port": 57621, "dest_ip": "192.168.21.255", "dest_port": 57621, "proto": "UDP", "alert": { "action": "allowed", "gid": 1, "signature_id": 2027397, "rev": 1, "signature": "ET POLICY Spotify P2P Client", "category": "Not Suspicious Traffic", "severity": 3, "metadata": { "updated_at": [ "2019_05_30" ], "performance_impact": [ "Low" ], "created_at": [ "2019_05_30" ], "signature_severity": [ "Minor" ], "deployment": [ "Internal" ], "attack_target": [ "Client_Endpoint" ], "affected_product": [ "Windows_Client_Apps" ] } }, "app_proto": "failed", "flow": { "pkts_toserver": 1, "pkts_toclient": 0, "bytes_toserver": 86, "bytes_toclient": 0, "start": "2019-05-31T10:18:40.125923-0300" }, "payload": "U3BvdFVkcDBLCKgSAE27EAABAARIlcIDo3coC8mysGtyrPsNoy5gZZI4DZY=", "payload_printable": "SpotUdp0K....M......H....w(....kr..\r..
e.8\r.",
"stream": 0,
"packet": "////////aPcoseVRCABFAABI43VAAEARqlvAqBWEwKgV/+EV4RUANPg3U3BvdFVkcDBLCKgSAE27EAABAARIlcIDo3coC8mysGtyrPsNoy5gZZI4DZY=",
"packet_info": {
"linktype": 1
}
}
selks-user@SELKS:/var/log/suricata$
I see that it started to detect one of the new recently added rules of Spotify, but I still do not see others that I generate on my own (Torrents, Blacksun, malware)
Ok- so the logging is functional/up and running. Now you we need to concentrate on the case that it does not alert and investigate why. Case by case to start with would be preferable I suggest.
I have made a clean installation in an ESXi, I have configured everything according to instructions. I see that there are packages received by the EVENTBOX, but I do not have data in the main Dashboard.
I have already configured my networks and interfaces but I have not been successful.
Can someone guide me with this topic?
attach a series of captures