search

ntop / ntopng

Web-based Traffic and Security Network Traffic Monitoring
http://www.ntop.org
GNU General Public License v3.0
6.23k stars 654 forks source link

Companion Interface option doesn't exist anymore #8746

Closed qti-admin closed 1 week ago

qti-admin commented 2 weeks ago

Environment:

What happened: Hi, I'm Integring Suricata with Ntop-ng, following the steps in the manual: https://www.ntop.org/guides/ntopng/third_party_integrations/suricata.html But when I try to link suricata alerts to ntopng, I need to select the syslog interface in the option "Companion Interface", but this option doesn't exists in GUI. The first thing that I notice is syslog interface is not recognized by ntop: image

How did you reproduce it? With ethernet card selected, navigate to Menu -> Interface -> Details Click on Settings button Enable "Mirrored Traffic" button image And i stuck here, because the Companion Interface option doesn't appear.

Debug Information: oct 05 03:15:19 auditor-red systemd[1]: Starting ntopng.service - ntopng high-speed web-based traffic monitoring and analysis tool... oct 05 03:15:19 auditor-red systemd[1]: Started ntopng.service - ntopng high-speed web-based traffic monitoring and analysis tool. oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [Redis.cpp:167] Successfully connected to redis 127.0.0.1:6379@0 oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [Redis.cpp:167] Successfully connected to redis 127.0.0.1:6379@0 oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:371] [LICENSE] No license file found /etc/ntopng.license: reading license from redis oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:545] [LICENSE] Unable to validate license [Empty license file] oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:696] WARNING: [LICENSE] Invalid license [Empty license file] oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:696] WARNING: [LICENSE] Invalid license [Empty license file] oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:725] WARNING: [LICENSE] ntopng will now run in Enterprise XL edition for 10 minutes oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:725] WARNING: [LICENSE] ntopng will now run in Enterprise XL edition for 10 minutes oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:731] WARNING: [LICENSE] before returning to community mode oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:731] WARNING: [LICENSE] before returning to community mode oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:733] WARNING: [LICENSE] You can buy a permanent license at http://shop.ntop.org oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:733] WARNING: [LICENSE] You can buy a permanent license at http://shop.ntop.org oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:737] WARNING: [LICENSE] or run ntopng in community mode starting oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:737] WARNING: [LICENSE] or run ntopng in community mode starting oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:739] WARNING: [LICENSE] ntopng --community oct 05 03:15:19 auditor-red ntopng[1238]: 05/Oct/2024 03:15:19 [NtopPro.cpp:739] WARNING: [LICENSE] ntopng --community oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [PF_RINGInterface.cpp:77] Reading packets from PF_RING v.8.9.0 interface enp1s0... oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [PF_RINGInterface.cpp:373] Packet capture filter set to "not net 172.20.17.0/24" oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Ntop.cpp:2839] Registered interface 'enp1s0' [id: 2] oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [SyslogCollectorInterface.cpp:36] Starting UDP syslog collector on 127.0.0.1:9999 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [SyslogCollectorInterface.cpp:80] Accepting UDP connections on 127.0.0.1:9999 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [SyslogCollectorInterface.cpp:36] Starting TCP syslog collector on 127.0.0.1:9999 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [SyslogCollectorInterface.cpp:66] ERROR: bind error oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [SyslogCollectorInterface.cpp:66] ERROR: bind error oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [NetworkInterface.cpp:3849] Cleanup interface syslog://127.0.0.1:9999 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [main.cpp:276] Unable to open interface syslog://127.0.0.1:9999. Falling back to pcap. oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [NetworkInterface.cpp:3849] Cleanup interface syslog://127.0.0.1:9999 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [main.cpp:289] ERROR: Unable to open interface syslog://127.0.0.1:9999 with pcap [19]: No such device oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [main.cpp:289] ERROR: Unable to open interface syslog://127.0.0.1:9999 with pcap [19]: No such device oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [main.cpp:373] PID stored in file /var/run/ntopng.pid oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Geolocation.cpp:172] Loaded database dbip-city-lite.mmdb [/usr/share/ntopng/httpdocs/geoip//dbip-city-lite.mmdb][ip_version: 6] oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Geolocation.cpp:172] Loaded database dbip-asn-lite.mmdb [/usr/share/ntopng/httpdocs/geoip//dbip-asn-lite.mmdb][ip_version: 6] oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Geolocation.cpp:111] Using geolocation provided by DB-IP (https://db-ip.com) oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Utils.cpp:857] User changed to ntopng oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [HTTPserver.cpp:1664] Found TLS certificate /usr/share/ntopng/httpdocs/ssl/ntopng-cert.pem oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [HTTPserver.cpp:1966] Web server dirs [/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts] oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [HTTPserver.cpp:1971] HTTP server listening on 3000 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [NetworkInterface.cpp:3576] Started flow user script hooks loop on interface 'enp1s0' [id: 2]... oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [NetworkInterface.cpp:3635] Started host user script hooks loop on interface 'enp1s0' [id: 2]... oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [main.cpp:463] Working directory: /var/lib/ntopng oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [main.cpp:465] Scripts/HTML pages directory: /usr/share/ntopng oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Ntop.cpp:553] Welcome to ntopng x86_64 v.6.3.241004 (dev:e4632b38485e9ef5caa13506305701d2bd3a0dd5:20241004) oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Ntop.cpp:562] Built on Ubuntu 24.04.1 LTS oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Ntop.cpp:564] (C) 1998-24 ntop oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [NtopPro.cpp:963] [LICENSE] System Id: LA37B7FF6920CAB21--UA37B7FF63B827B5F--OL oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [NtopPro.cpp:965] [LICENSE] Edition: Community oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [NtopPro.cpp:967] [LICENSE] Lic. Type: Time-Limited [Empty license file] License oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [NtopPro.cpp:1000] [LICENSE] Validity: Until Sat Oct 5 03:25:19 2024 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [FlowRiskAlerts.cpp:276] [!] nDPI risk 56/Obfuscated Traffic has not been defined in ntopng oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Ntop.cpp:997] Adding 172.20.10.62/32 as IPv4 interface address for enp1s0 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Ntop.cpp:1008] Adding 172.20.10.0/24 as IPv4 local network for enp1s0 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Ntop.cpp:1033] Adding fe80::96c6:91ff:fe78:79f4/128 as IPv6 interface address for enp1s0 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [Ntop.cpp:1045] Adding fe80::96c6:91ff:fe78:79f4/64 as IPv6 local network for enp1s0 oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [PeriodicActivities.cpp:122] Started periodic activities loop... oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [startup.lua:38] Processing startup.lua: please hold on... oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [startup.lua:42] [asset_inventory.lua:49] [Asset Inventory] Initalization... oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [startup.lua:42] [asset_inventory_db.lua:23] [Asset Inventory DB] Initialization completed oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:803] Refreshing category lists... oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:594] Loaded Abuse.ch URLhaus: 262 rules oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:594] Loaded Emerging Threats: 1371 rules oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:594] Loaded IPsum Threat Intelligence Feed: 32528 rules oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:594] Loaded NoCoin Filter List: 409 rules oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:594] Loaded SSLBL Botnet C2 IP Blacklist: 32 rules oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:594] Loaded Stratosphere Lab: 12325 rules oct 05 03:15:21 auditor-red ntopng[1238]: [192B blob data] oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:594] Loaded ThreatFox: 9047 rules oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:594] Loaded dshield 7 days: 29 rules oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:178] [lists_utils.lua:700] Loaded Category Lists (9718 hosts, 46285 IPs) loaded in 0 sec oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:182] Initializing device polices... oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:198] Initializing alerts... oct 05 03:15:21 auditor-red ntopng[1238]: 05/Oct/2024 03:15:21 [startup.lua:207] Initializing timeseries... oct 05 03:15:22 auditor-red ntopng[1238]: 05/Oct/2024 03:15:22 [startup.lua:316] Completed startup.lua oct 05 03:15:22 auditor-red ntopng[1238]: 05/Oct/2024 03:15:22 [NetworkInterface.cpp:3812] Started packet polling on interface 'enp1s0' [id: 2]... oct 05 03:15:32 auditor-red sudo[1552]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl n2disk-ntopng stats enp1s0 oct 05 03:15:32 auditor-red sudo[1552]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=990) oct 05 03:15:32 auditor-red sudo[1552]: pam_unix(sudo:session): session closed for user root oct 05 03:15:32 auditor-red sudo[1561]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl disk2disk-ntopng stats enp1s0 oct 05 03:15:32 auditor-red sudo[1561]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=990) oct 05 03:15:32 auditor-red sudo[1561]: pam_unix(sudo:session): session closed for user root oct 05 03:16:03 auditor-red sudo[1699]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl n2disk-ntopng stats enp1s0 oct 05 03:16:03 auditor-red sudo[1699]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=990) oct 05 03:16:03 auditor-red sudo[1699]: pam_unix(sudo:session): session closed for user root oct 05 03:16:04 auditor-red sudo[1708]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl disk2disk-ntopng stats enp1s0 oct 05 03:16:04 auditor-red sudo[1708]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=990) oct 05 03:16:04 auditor-red sudo[1708]: pam_unix(sudo:session): session closed for user root oct 05 03:32:08 auditor-red sudo[2944]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl n2disk-ntopng stats enp1s0 oct 05 03:32:08 auditor-red sudo[2944]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=990) oct 05 03:32:08 auditor-red sudo[2944]: pam_unix(sudo:session): session closed for user root oct 05 03:32:08 auditor-red sudo[2953]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl disk2disk-ntopng stats enp1s0 oct 05 03:32:08 auditor-red sudo[2953]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=990) oct 05 03:32:08 auditor-red sudo[2953]: pam_unix(sudo:session): session closed for user root

amanzur76 commented 2 weeks ago

Hello I am facing the exact same issue

MatteoBiscosi commented 1 week ago

Hi the issue is that you have problems when opening the syslog interface, see in the logs you sent:

oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [SyslogCollectorInterface.cpp:66] ERROR: bind error oct 05 03:15:20 auditor-red ntopng[1238]: 05/Oct/2024 03:15:20 [SyslogCollectorInterface.cpp:66] ERROR: bind error

The companion interface is available only if you are analyzing at least 2 interfaces otherwise the option will not appear. Are you sure that the syslog interface you tried to connect to is available on that port, ecc. ?

qti-admin commented 1 week ago

Hi @MatteoBiscosi How could I check that? I checked the port 9999 that was suggested by ntop to use:

sudo lsof -i :9999 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME ntop-lice 18276 root 6u IPv4 50670 0t0 TCP *:9999 (LISTEN) ntopng-ma 39922 ntopng 24u IPv4 86091 0t0 UDP *:9999

MatteoBiscosi commented 1 week ago

Hi @qti-admin could you please share your ntopng configuration file?

qti-admin commented 1 week ago

``

soporte@auditor-red:~$ grep ^[^#] /etc/ntopng/ntopng.conf
-G=/var/run/ntopng.pid
-i=enp1s0
-w=3000
-i=syslog://*:9999
cardigliano commented 1 week ago

@qti-admin , this is working in our lab, we need to dig more on your installation, the bind error means the port is somehow already used..

qti-admin commented 1 week ago

Hi @cardigliano, thanks for your help, here are my answers:

But, is this an expected behaviour? I'm asking this, because I followed the instructions in a clean OS, so I think that's weird behaviour. https://www.ntop.org/guides/ntopng/third_party_integrations/suricata.html#suricata-integration So, how could I check what service or application is creating this conflict?

Appreciate your help.

cardigliano commented 1 week ago

My guess was that port 9999 was already in use, however lsof does not show it which is weird. Anyway the fact that now is working means there was actually a conflict. Please keep using a different port (9999 was just a number used in the doc example).