search

ntop / ntopng

Web-based Traffic and Security Network Traffic Monitoring
http://www.ntop.org
GNU General Public License v3.0
6.32k stars 658 forks source link

ntopng GUI stopped working, cannot log in #4835

Closed bengt-a closed 3 years ago

bengt-a commented 3 years ago

Hi, Running ntopng on a Raspberry Pi 3+ with the latest version for some weeks, I ran a sudo apt-get update and upgrade. After this, and possibly some other work, ntopng suddenly changed from the nice dark theme to a broken-looking HTML page on the already open pages. And if I try to log in, I never get past the login page. Entering any username and password just clears the fields. I have tried both with the password I set, and with password "admin"

I followed every step in the guide on https://www.ntop.org/guides/ntopng/faq.html#cannot-login-into-the-gui, but that didn't help. 

redis-cli set ntopng.prefs.local.auth_enabled 1
redis-cli del ntopng.user.admin.password
for k in `redis-cli keys ntopng.cache.failed_logins.*`; do redis-cli del $k; done
sudo systemctl stop ntopng
redis-cli keys "ntopng.*" | xargs redis-cli unlink
sudo rm /var/lib/ntopng/runtimeprefs.json
sudo systemctl restart ntopng

NB the command sudo rm /var/lib/ntopng/runtimeprefs.json didn't work (file not found), and when doing a full find on my pi that file doesn't seem to exist (pi@trafficPi3 ~ $ sudo find / -name runtimeprefs.json -print yields nothing. Also not if I search only in /var or /etc)

I am running the latest version of ntopng (before the crash it showed 4.3.201207 (12418) - Community/Embedded Edition) I start ntopng with sudo systemctl restart ntopng Raspbian version:

pi@trafficPi3 ~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"

I also tried sudo apt-get install --reinstall ntopng but the problem still persists.

Some output from the program ntopng-config I just found:

                     ┌────────────────────────────────────────────┤ Report ├────────────────────────────────────────────┐
                     │                                                                                                  │
                     │ ● ntopng.service - ntopng high-speed web-based traffic monitoring and analysis tool
                     │    Loaded: loaded (/etc/systemd/system/ntopng.service; enabled; vendor preset: enabled)
                     │    Active: active (running) since Mon 2020-12-14 19:01:13 CET; 8min ago
                     │   Process: 934 ExecStartPre=/bin/sh -c /usr/bin/ntopng-utils-manage-config -a check-restore  &&
                     │ /usr/bin/ntopng-utils-manage-config -a restore || true (code=exited, status=0/SUCCESS)
                     │   Process: 954 ExecStartPre=/bin/sh -c /bin/cat /etc/ntopng/ntopng.conf > /run/ntopng.conf.raw
                     │ (code=exited, status=0/SUCCESS)
                     │   Process: 956 ExecStartPre=/bin/sh -c /bin/cat /etc/ntopng/ntopng.conf.d/*.conf >>
                     │ /run/ntopng.conf.raw 2>/dev/null || true (code=exited, status=0/SUCCESS)
                     │   Process: 958 ExecStartPre=/bin/sh -c /bin/sed "/^[ ]*-e.*$\|^[ ]*-G.*\|^[ ]*--daemon.*\|[
                     │ ]*--pid.*/s/^/#/" /run/ntopng.conf.raw > /run/ntopng.conf (code=exited, status=0/SUCCESS)
                     │  Main PID: 960 (ntopng)
                     │     Tasks: 43 (limit: 2063)
                     │    CGroup: /system.slice/ntopng.service
                     │            └─960 /usr/bin/ntopng /run/ntopng.conf
                     │
                     │ Dec 14 19:01:41 trafficPi3 ntopng[960]: 14/Dec/2020 19:01:41 [startup.lua:165]
                     │ [lists_utils.lua:406] Updating list 'NoCoin Filter List'

Version info:

pi@trafficPi3 ~ $ sudo ntopng --version
v.4.3.201213    [Enterprise/Professional/Embedded build]
GIT rev:        dev:1b950cd93b7d925ae1162ea67538c239ead3ce29:20201213
Pro rev:        r3400
Built on:       Raspbian GNU/Linux 10 (buster)
System Id:      FD0C4FA1499602D2
Platform:       armv7l
Edition:        Enterprise Embedded
License Type:   Time-Limited [Empty license file]
Validity:       Until Mon Dec 14 19:23:15 2020

Trying to Edit the ntopng.conf with sudo ntopng-utils-manage-config, when I check Flows, Hosts, Networks and Interfaces, it says "no value found" Not sure what to do next.

simonemainardi commented 3 years ago

please, attach screenshots, and also possible browser console errors you may see.

bengt-a commented 3 years ago

image

This is all I get since I cannot log in.

Console log: GET http://192.168.1.18:3000/bootstrap-4.4.0-dist/css/bootstrap.min.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:8 GET http://192.168.1.18:3000/css/minified/fontawesome-custom.min.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:9 GET http://192.168.1.18:3000/css/minified/tempusdominus.min.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:10 GET http://192.168.1.18:3000/css/minified/heatmap.min.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:11 GET http://192.168.1.18:3000/css/minified/rickshaw.min.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:12 GET http://192.168.1.18:3000/css/dc.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:13 GET http://192.168.1.18:3000/selectpicker/css/bootstrap-select.min.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:23 GET http://192.168.1.18:3000/css/flags.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:24 GET http://192.168.1.18:3000/css/pie-chart.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:26 GET http://192.168.1.18:3000/css/nv.d3.css net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:27 GET http://192.168.1.18:3000/css/custom_theme.css?1607969542 net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:31 GET http://192.168.1.18:3000/js/jquery_bootstrap.min.js?1607969542 net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:32 GET http://192.168.1.18:3000/popper-1.12.9/js/popper.min.js net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:33 GET http://192.168.1.18:3000/bootstrap-4.4.0-dist/js/bootstrap.min.js?1607969542 net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:34 GET http://192.168.1.18:3000/js/deps.min.js?1607969542 net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:35 GET http://192.168.1.18:3000/js/ntop.min.js?1607969542 net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:36 GET http://192.168.1.18:3000/js/tempusdominus.min.js?1607969542 net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:104 Uncaught ReferenceError: $ is not defined at login.lua?referer=192.168.1.18%3A3000%2F:104 (anonymous) @ login.lua?referer=192.168.1.18%3A3000%2F:104 login.lua?referer=192.168.1.18%3A3000%2F:37 GET http://192.168.1.18:3000/selectpicker/js/bootstrap-select.min.js?1607969542 net::ERR_ABORTED 404 (Not Found) login.lua?referer=192.168.1.18%3A3000%2F:1 Unchecked runtime.lastError: Could not establish connection. Receiving end does not exist.

simonemainardi commented 3 years ago

I am experiencing normal behavior with 

root@pi3:/storage/home/mainardi# ntopng --version
v.4.3.201213    [Enterprise/Professional/Embedded build]
GIT rev:    dev:1b950cd93b7d925ae1162ea67538c239ead3ce29:20201213
Pro rev:    r3400
Built on:   Raspbian GNU/Linux 10 (buster)
System Id:  FD504ACA499602D2
Platform:   armv7l

image

You should not touch sudo ntopng-utils-manage-config manually.

Yours seems more a browser issue. There's an ERR_ABORTED as is someone was preventing requests from being performed. Don't know why you are getting this. I would try and

bengt-a commented 3 years ago

I also suspected the browser as a possible culprit, and I have tried with two different browsers (Chrome and Edge), on two different computers. Same result unfortunately.

Please note that it was working fine for a long time. Then suddenly, at the same time as I did apt-get upgrade, it stopped working properly.

If I completely wipe the installation and reinstall, what config files are e.g. the custom IP/MAC associations stored in so I can save them before doing the wipe?

simonemainardi commented 3 years ago

do you want me to connect and do a check before wiping?

bengt-a commented 3 years ago

That would have been great! I was available last evening (CET). It seems like we are in totally different time zones, or at least available at different times.

Please can you post instructions on where data is stored for IP/MAC associations etc, and detailed instructions on how to completely reinstall the software? (I tried apt-get purge ntopng and reinstalled it, but there was no effect.) I also cleaned out /var/lib/ntopng. Still no effect.

bengt-a commented 3 years ago

Anyone? I suspect the web server files are corrupt. How to reinstall? Or if no information, how do I completely remove/purge the application and start over from scratch?

bengt-a commented 3 years ago

I tried completely uninstalling and reinstalling. No change.

I am starting to suspect the installation of Monitorix on the same server. It is apparently using a httpd server.

When checking with netstat -a, I see monitorix listening on port 3030. Could this httpd server be interfering with ntopng somehow? What can I do to sort this out?

simonemainardi commented 3 years ago

you need to understand why you get errors ERR_ABORTED 404, Could not establish connection. Receiving end does not exist.

It could be a firewall preventing ntopng to be reached. Check the presence of firewalls, check iptables, and also remove Monitorix if you have the suspect it can be the culprit.

Again ntopng works OK on the pi3 so it must be something affecting your installation only.

bengt-a commented 3 years ago

Hello! I have made no changes to my infrastructure, and there is no firewall. Like I said, all I have done is upgrade ntopng via apt-get upgrade. How is ntopng serving its webpages? It seems like it's the web server which is having severe issues.

I already explained that it has already worked on my Pi3, but suddenly stopped working. So yes, it is probably affecting my installation only. This is why I am reaching out to you for support since I am trying to troubleshoot the issue.

On Fri, Dec 18, 2020 at 2:20 PM Simone Mainardi notifications@github.com wrote:

you need to understand why you get errors ERR_ABORTED 404, Could not establish connection. Receiving end does not exist.

It could be a firewall preventing ntopng to be reached. Check the presence of firewalls, check iptables, and also remove Monitorix if you have the suspect it can be the culprit.

Again ntopng works OK on the pi3 so it must be something affecting your installation only.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ntop/ntopng/issues/4835#issuecomment-748080426, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7UJP4DNHIRYP5PVPDASV3SVNJKBANCNFSM4U3EQIPA .

simonemainardi commented 3 years ago

I am starting to suspect the installation of Monitorix on the same server. It is apparently using a httpd server.

please try and remove it

bengt-a commented 3 years ago

I checked with Monitorix support and don't see how the tools could interfere with each other since they are using separate ports.

Please instead explain how the ntopng web server is working and configured. Where are the files stored? Where is the configuration file? Where is the log file?

With this information I can troubleshoot further.

PS. I also disabled the login optiojn via the --disable-login 1 flag. The result can be seen in the next post:

Is there some debugging log you need? I can't figure out where the ntopng error log is.

Cheers, Bengt

On Fri, Dec 18, 2020, 19:00 Simone Mainardi notifications@github.com wrote:

I am starting to suspect the installation of Monitorix on the same server. It is apparently using a httpd server.

please try and remove it

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ntop/ntopng/issues/4835#issuecomment-748233488, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7UJPZRTWKX6YU6NF6622LSVOKFJANCNFSM4U3EQIPA .

bengt-a commented 3 years ago

image

bengt-a commented 3 years ago

I see in the browser console that I am getting lots of errors: Failed to load resource: the server responded with a status of 404 (Not Found) nv.d3.css:1 Failed to load resource: the server responded with a status of 404 (Not Found) custom_theme.css:1 Failed to load resource: the server responded with a status of 404 (Not Found) dashboard.lua:578 Uncaught ReferenceError: $ is not defined at getData (dashboard.lua:578) getData @ dashboard.lua:578 dashboard.lua:600 Uncaught ReferenceError: $ is not defined at getData (dashboard.lua:600) getData @ dashboard.lua:600 2dashboard.lua:622 Uncaught ReferenceError: $ is not defined at getData (dashboard.lua:622)

This is quite frustrating...

bengt-a commented 3 years ago

I think I have found the fault now! /usr/local/share/ntopng/httpdocs/ is empty!

pi@trafficPi3 /usr/local/share/ntopng/httpdocs $ ls
geoip
pi@trafficPi3 /usr/local/share/ntopng/httpdocs $

I can see when running ntopng in debug mode that it is trying to serve pages from this directory:

18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/bootstrap-4.4.0-dist/css/bootstrap.min.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/minified/ntopng.min.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/minified/tempusdominus.min.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/minified/fontawesome-custom.min.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/minified/heatmap.min.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/minified/rickshaw.min.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/custom_theme.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/dc.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/flags.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/selectpicker/css/bootstrap-select.min.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/pie-chart.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/js/jquery_bootstrap.min.js
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/css/nv.d3.css
18/Dec/2020 22:39:57 [HTTPserver.cpp:1285] [HTTP] Serving file /usr/local/share/ntopng/httpdocs/bootstrap-4.4.0-dist/js/bootstrap.min.js

So... how can I regenerate the files which should go in this directory? Why haven't they been generated when I reinstalled the program?

bengt-a commented 3 years ago

I solved the issue together with a friend. Apparently the httpdocs are located in /usr/share/ntopng, but ntopng looks in /usr/local/share/ntopng by default. Starting ntopng with flag --httpdocs-dir /usr/share/ntopng/httpdocs solved the issue.

I appreciate that I am running ntopng as a free service, but I am a little surprised that nobody in the community, or in the support, could even point me in this direction but only suggested the preposterous idea I myself suggested that some other software was interfering.

One last hope of support: Is there some way I can recover my old preferences and settings, such as Mac address translations?

simonemainardi commented 3 years ago

I solved the issue together with a friend. Apparently the httpdocs are located in /usr/share/ntopng, but ntopng looks in /usr/local/share/ntopng by default. Starting ntopng with flag --httpdocs-dir /usr/share/ntopng/httpdocs solved the issue.

Thanks for troubleshooting this.

Issue (fixed) was due a wrong lookup of paths, causing the default path to be taken as /local, that is, the default path selected when ./configure is used without --prefix.

Future builds won't even need you to specify --httpdocs-dir /usr/share/ntopng/httpdocs.

Anyway I am not sure why your directory was empty. It should have not been present at all.

I appreciate that I am running ntopng as a free service, but I am a little surprised that nobody in the community, or in the support, could even point me in this direction but only suggested the preposterous idea I myself suggested that some other software was interfering.

If you suggest an idea, community and support will believe you are smart enough to suggest grounded ideas and verify them.

One last hope of support: Is there some way I can recover my old preferences and settings, such as Mac address translations?

If you've wiped redis, no. But you can do a backup of preferences from "Manage Data" that can be used in the future.

bengt-a commented 3 years ago

Hello Simone, Thank you for your response!

I am happy that it is working now. I will remember to back up my data going forward, and I will be looking forward to the future builds where it is not necessary to specify the httpdocs dir.

Merry Christmas and happy holidays, Bengt

On Wed, Dec 23, 2020 at 5:14 PM Simone Mainardi notifications@github.com wrote:

I solved the issue together with a friend. Apparently the httpdocs are located in /usr/share/ntopng, but ntopng looks in /usr/local/share/ntopng by default. Starting ntopng with flag --httpdocs-dir /usr/share/ntopng/httpdocs solved the issue.

Thanks for troubleshooting this.

Issue (fixed) was due a wrong lookup of paths, causing the default path to be taken as /local, that is, the default path selected when ./configure is used without --prefix.

Future builds won't even need you to specify --httpdocs-dir /usr/share/ntopng/httpdocs.

Anyway I am not sure why your directory was empty. It should have not been present at all.

I appreciate that I am running ntopng as a free service, but I am a little surprised that nobody in the community, or in the support, could even point me in this direction but only suggested the preposterous idea I myself suggested that some other software was interfering.

If you suggest an idea, community and support will believe you are smart enough to suggest grounded ideas and verify them.

One last hope of support: Is there some way I can recover my old preferences and settings, such as Mac address translations?

If you've wiped redis, no. But you can do a backup of (future) preferences from "Manage Data" that can be used in the future.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ntop/ntopng/issues/4835#issuecomment-750368887, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7UJPZLQIK2ZLIAYGPJTZ3SWIJMZANCNFSM4U3EQIPA .