search

ntop / ntopng

Web-based Traffic and Security Network Traffic Monitoring
http://www.ntop.org
GNU General Public License v3.0
6.26k stars 657 forks source link

ntopng UI is extremly slow #8209

Closed goltob closed 9 months ago

goltob commented 9 months ago

Environment:

What happened: Opening the ntopng interface is extremely slow. It takes up to 3 minutes after login for the page to load.

RAM: 32 GB 32 Core Intex Xeon 2.6 GHz

How did you reproduce it?

Debug Information:

journalctl -u ntopng output:

Feb 09 09:26:31 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:31 [Ntop.cpp:993] Adding fe80::1a66:daff:fef5:e90c/64 as IPv6 local network for eno1 Feb 09 09:26:31 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:31 [PeriodicActivities.cpp:108] Started periodic activities loop... Feb 09 09:26:31 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:31 [startup.lua:38] Processing startup.lua: please hold on... Feb 09 09:26:32 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:32 [startup.lua:121] [lists_utils.lua:825] Category lists not loaded (offline) Feb 09 09:26:32 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:32 [startup.lua:121] [lists_utils.lua:466] Failure loading host 'htttt' category '100' in list 'Abuse.ch URLhaus' Feb 09 09:26:32 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:32 [startup.lua:121] [lists_utils.lua:679] List 'Stratosphere Lab' has 0 rules. Please report this to https://github.com/ntop/ntopng Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:121] [lists_utils.lua:466] Failure loading host '56565' category '100' in list 'ThreatFox' Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:121] [lists_utils.lua:466] Failure loading host 'noluyoruzawk' category '100' in list 'ThreatFox' Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:121] [lists_utils.lua:466] Failure loading host '4040' category '100' in list 'ThreatFox' Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:121] [lists_utils.lua:466] Failure loading host 'datacikerim' category '100' in list 'ThreatFox' Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:121] [lists_utils.lua:466] Failure loading host 'nicehash' category '100' in list 'ThreatFox' Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:121] [lists_utils.lua:466] Failure loading host 'mpapwpodllalw' category '100' in list 'ThreatFox' Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:121] [lists_utils.lua:753] Category Lists (30013 hosts, 1086 IPs, 0 JA3) loaded in 1 sec Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:125] Initializing device polices... Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:141] Initializing alerts... Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:150] Initializing timeseries... Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [startup.lua:242] Completed startup.lua Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [PeriodicActivities.cpp:167] Found 10 activities Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [NetworkInterface.cpp:3720] Started packet polling on interface eno4 [id: 0]... Feb 09 09:26:33 SERVERNAME ntopng[470947]: 09/Feb/2024 09:26:33 [NetworkInterface.cpp:3720] Started packet polling on interface eno1 [id: 1]... Feb 09 09:41:06 SERVERNAME sudo[477417]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl n2disk-ntopng stats eno4 Feb 09 09:41:06 SERVERNAME sudo[477417]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=996) Feb 09 09:41:06 SERVERNAME sudo[477417]: pam_unix(sudo:session): session closed for user root Feb 09 09:41:06 SERVERNAME sudo[477426]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl disk2disk-ntopng stats eno4 Feb 09 09:41:06 SERVERNAME sudo[477426]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=996) Feb 09 09:41:06 SERVERNAME sudo[477426]: pam_unix(sudo:session): session closed for user root Feb 09 09:43:29 SERVERNAME sudo[478474]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl n2disk-ntopng stats eno4 Feb 09 09:43:29 SERVERNAME sudo[478474]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=996) Feb 09 09:43:29 SERVERNAME sudo[478474]: pam_unix(sudo:session): session closed for user root Feb 09 09:43:29 SERVERNAME sudo[478483]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl disk2disk-ntopng stats eno4 Feb 09 09:43:29 SERVERNAME sudo[478483]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=996) Feb 09 09:43:29 SERVERNAME sudo[478483]: pam_unix(sudo:session): session closed for user root Feb 09 09:45:53 SERVERNAME ntopng[470947]: [LINT] validateParameter failed for parameter [network][function] Feb 09 09:45:53 SERVERNAME ntopng[470947]: string -1

systemctl status ntopng ● ntopng.service - ntopng high-speed web-based traffic monitoring and analysis tool Loaded: loaded (/etc/systemd/system/ntopng.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/ntopng.service.d └─nbox.conf Active: active (running) since Fri 2024-02-09 09:26:30 CET; 22min ago Process: 470927 ExecStartPre=/bin/sh -c /usr/bin/ntopng-utils-manage-config -a check-restore && /usr/bin/ntopng-utils-manage-config -a restore || true (code=exited, status=0/SUCCESS) Process: 470941 ExecStartPre=/bin/sh -c /bin/cat /etc/ntopng/ntopng.conf > /run/ntopng.conf.raw (code=exited, status=0/SUCCESS) Process: 470943 ExecStartPre=/bin/sh -c /bin/cat /etc/ntopng/ntopng.conf.d/.conf >> /run/ntopng.conf.raw 2>/dev/null || true (code=exited, status=0/SUCCESS) Process: 470945 ExecStartPre=/bin/sh -c /bin/sed "/^[ ]-e.$|^[ ]-G.|^[ ]--daemon.|[ ]--pid.*/s/^/#/" /run/ntopng.conf.raw > /run/ntopng.conf (code=exited, status=0/SUCCESS) Process: 470948 ExecStartPost=/usr/bin/node /usr/share/nbox/scripts/nbox-notify.js ntopng start (code=exited, status=0/SUCCESS) Main PID: 470947 (ntopng-main) Tasks: 33 (limit: 77020) Memory: 409.7M CPU: 4min 6.356s CGroup: /system.slice/ntopng.service └─470947 /usr/bin/ntopng /run/ntopng.conf

Feb 09 09:41:06 sudo[477426]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=996) Feb 09 09:41:06 sudo[477426]: pam_unix(sudo:session): session closed for user root Feb 09 09:43:29 SERVERNAME sudo[478474]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl n2disk-ntopng stats eno4 Feb 09 09:43:29 SERVERNAME sudo[478474]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=996) Feb 09 09:43:29 SERVERNAME sudo[478474]: pam_unix(sudo:session): session closed for user root Feb 09 09:43:29 SERVERNAME sudo[478483]: ntopng : PWD=/ ; USER=root ; COMMAND=/usr/bin/ntopctl disk2disk-ntopng stats eno4 Feb 09 09:43:29 SERVERNAME sudo[478483]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=996) Feb 09 09:43:29 SERVERNAME sudo[478483]: pam_unix(sudo:session): session closed for user root Feb 09 09:45:53 SERVERNAME ntopng[470947]: [LINT] validateParameter failed for parameter [network][function] Feb 09 09:45:53 SERVERNAME ntopng[470947]: string -1

cat /etc/ntopng/ntopng.conf -i=eno4 -i=eno1 -G=/var/run/ntopng.pid

-F="clickhouse;127.0.0.1;ntopng;default;xxxxx"

-m="10.151.238.0/24, 10.152.38.0/23, 10.152.72.2/23, 192.168.0.0/16, 172.28.254.0/20, 172.29.0.0/24, 172.27.0.0/24, 172.30.20.0/24" -n=1 -p=/var/lib/ntopng/protos.txt --offline

goltob commented 9 months ago

It seems that I was able to solve it. You have to know that I have no DNS. So the problem (probably) was that the /etc/hosts file was configured incorrectly.

lucaderi commented 9 months ago

Thanks for your information