search

ntop / ntopng

Web-based Traffic and Security Network Traffic Monitoring
http://www.ntop.org
GNU General Public License v3.0
6.2k stars 650 forks source link

Missing CSS sheets in 5.4 build for UDM Pro #6805

Closed tusc closed 2 years ago

tusc commented 2 years ago

Environment:

What happened: I maintain the port of ntopng forthe Ubiquiti series of routers (UDM/UDM Pro) located here: https://github.com/tusc/ntopng-udm

I've been trying to update to the latest stable release of ntopng, 5.4. I've been able to build the packages without issue. The issue is trying login after startup. I see a login page without any css sheets. I have to disable login in order to get to the main dashboard. It sounds similar to this issue: https://github.com/ntop/ntopng/issues/6333

I see something similar to the following at login: image

Debug Information:

Paste debug information below:

From the main dashboard I'm seeing this: image

Output from ntopng shows nothing unusual:


root@UDMPro:/# ntopng /etc/ntopng/ntopng.conf
30/Jul/2022 17:08:30 [Ntop.cpp:3600] WARNING: Unable to find timezone: using UTC
30/Jul/2022 17:08:30 [Prefs.cpp:1311] All HTTP user login disabled
30/Jul/2022 17:08:30 [Redis.cpp:151] Successfully connected to redis 127.0.0.1:6379@0
30/Jul/2022 17:08:30 [Redis.cpp:151] Successfully connected to redis 127.0.0.1:6379@0
30/Jul/2022 17:08:30 [boot.lua:24] WARNING: No connectivity detected, ntopng will run in offline mode
sh: 1: netstat: not found
30/Jul/2022 17:08:30 [PcapInterface.cpp:96] Reading packets from br0 [id: 0]
30/Jul/2022 17:08:30 [Ntop.cpp:2589] Registered interface br0 [id: 0]
30/Jul/2022 17:08:30 [main.cpp:337] PID stored in file /var/run/ntopng.pid
30/Jul/2022 17:08:30 [Geolocation.cpp:149] Loaded database dbip-city-lite.mmdb [/usr/share/ntopng/httpdocs/geoip//dbip-city-lite.mmdb][ip_version: 6]
30/Jul/2022 17:08:30 [Geolocation.cpp:149] Loaded database dbip-asn-lite.mmdb [/usr/share/ntopng/httpdocs/geoip//dbip-asn-lite.mmdb][ip_version: 6]
30/Jul/2022 17:08:30 [Geolocation.cpp:95] Using geolocation provided by DB-IP (https://db-ip.com)
30/Jul/2022 17:08:31 [HTTPserver.cpp:1408] Found TLS certificate /usr/share/ntopng/httpdocs/ssl/ntopng-cert.pem
30/Jul/2022 17:08:31 [HTTPserver.cpp:1683] Web server dirs [/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts]
30/Jul/2022 17:08:31 [HTTPserver.cpp:1686] HTTP server listening on 3000
30/Jul/2022 17:08:31 [HTTPserver.cpp:1692] HTTPS server listening on 3001
30/Jul/2022 17:08:31 [Utils.cpp:784] User changed to ntopng
30/Jul/2022 17:08:31 [NetworkInterface.cpp:3052] Started flow user script hooks loop on interface br0 [id: 0]...
30/Jul/2022 17:08:31 [NetworkInterface.cpp:3089] Started host user script hooks loop on interface br0 [id: 0]...
30/Jul/2022 17:08:31 [main.cpp:407] Working directory: /var/lib/ntopng
30/Jul/2022 17:08:31 [main.cpp:409] Scripts/HTML pages directory: /usr/share/ntopng
30/Jul/2022 17:08:31 [Ntop.cpp:466] Welcome to ntopng aarch64 v.5.4.220730 - (C) 1998-22 ntop.org
30/Jul/2022 17:08:31 [Ntop.cpp:879] Adding 192.168.1.1/32 as IPv4 interface address for br0
30/Jul/2022 17:08:31 [Ntop.cpp:888] Adding 192.168.1.0/24 as IPv4 local network for br0
30/Jul/2022 17:08:31 [Ntop.cpp:910] Adding fe80::10c4:b1ff:feeb:e14d/128 as IPv6 interface address for br0
30/Jul/2022 17:08:31 [Ntop.cpp:920] Adding fe80::10c4:b1ff:feeb:e14d/64 as IPv6 local network for br0
30/Jul/2022 17:08:31 [PeriodicActivities.cpp:109] Started periodic activities loop...
30/Jul/2022 17:08:32 [startup.lua:35] Processing startup.lua: please hold on...
30/Jul/2022 17:08:32 [startup.lua:120] [lists_utils.lua:807] Category lists not loaded (offline)
30/Jul/2022 17:08:32 [startup.lua:124] Initializing device polices...
30/Jul/2022 17:08:32 [startup.lua:140] Initializing alerts...
30/Jul/2022 17:08:32 [startup.lua:149] Initializing timeseries...
30/Jul/2022 17:08:32 [startup.lua:212] Fetching latest ntop blog posts...
30/Jul/2022 17:08:32 [startup.lua:216] Completed startup.lua
30/Jul/2022 17:08:32 [PeriodicActivities.cpp:167] Found 10 activities
30/Jul/2022 17:08:32 [NetworkInterface.cpp:3242] Started packet polling on interface br0 [id: 0]..
```.
tusc commented 2 years ago

Something has changed with this version. In the prior release which was running fine, the css directory was fully populated:

root@UDMPro:/usr/share/ntopng/httpdocs/css# ls -la
total 228
drwxr-xr-x  5 root root  4096 Dec  7  2020 .
drwxr-xr-x 21 root root  4096 Dec  7  2020 ..
-rw-r--r--  1 root root   833 Dec  6  2020 Makefile
-rw-r--r--  1 root root   124 Dec  6  2020 README.md
drwxr-xr-x  2 root root  4096 Dec  7  2020 Rickshaw
-rw-r--r--  1 root root  2982 Dec  6  2020 bootstrap-nav-wizard.css
-rwxr-xr-x  1 root root  7889 Dec  6  2020 bootstrap-slider.css
-rw-r--r--  1 root root  9038 Dec  6  2020 bootstrap3-datetimepicker.css
-rw-r--r--  1 root root   190 Dec  6  2020 bubble-chart.css
-rw-r--r--  1 root root  1025 Dec  6  2020 cal-heatmap.css
-rw-r--r--  1 root root   873 Dec  6  2020 cubism.css
-rw-r--r--  1 root root   570 Dec  6  2020 custom_theme.css
-rw-r--r--  1 root root  6384 Dec  6  2020 dark-mode.css
-rw-r--r--  1 root root  4340 Dec  6  2020 datepicker.css
-rw-r--r--  1 root root  4109 Dec  6  2020 dc.css
-rw-r--r--  1 root root 12737 Dec  6  2020 flags.css
-rw-r--r--  1 root root   336 Dec  6  2020 heatmap.css
drwxr-xr-x  2 root root  4096 Dec  7  2020 images
-rw-r--r--  1 root root 37326 Dec  6  2020 jquery-ui.css
drwxr-xr-x  2 root root  4096 Dec  7  2020 minified
-rw-r--r--  1 root root 16492 Dec  6  2020 ntopng.css
-rw-r--r--  1 root root  1434 Dec  6  2020 ntopng.min.css
-rw-r--r--  1 root root 12015 Dec  6  2020 nv.d3.css
-rw-r--r--  1 root root   999 Dec  6  2020 pie-chart.css
-rw-r--r--  1 root root  7309 Dec  6  2020 rickshaw.css
-rw-r--r--  1 root root   577 Dec  6  2020 sequence_sunburst.css
-rw-r--r--  1 root root  9016 Dec  6  2020 sidebar.css
-rw-r--r--  1 root root   890 Dec  6  2020 tablesorted.css
-rw-r--r--  1 root root  2040 Dec  6  2020 timeline.css
-rw-r--r--  1 root root  1789 Dec  6  2020 timepicker.css
root@UDMPro:/usr/share/ntopng/httpdocs/css# ntopng --version
v.4.2.201206   [Community build]
GIT rev:       HEAD:5e649a2d1130b4a3ab0c5bb673d615172cc0bdbb:20201206

Now in the 5.4 release, there are just two files. Did I miss a step during the build process?

root@UDMPro:/usr/share/ntopng/httpdocs/css# ls -la
total 28
drwxr-xr-x  2 root root  4096 Jul 30 16:49 .
drwxr-xr-x 13 root root  4096 Jul 30 16:49 ..
-rw-r--r--  1 root root   133 Jul 30 15:36 README.md
-rw-r--r--  1 root root 16265 Jul 30 15:36 select2.min.css
root@UDMPro:/usr/share/ntopng/httpdocs/css# ntopng --version
30/Jul/2022 19:57:25 [Ntop.cpp:3600] WARNING: Unable to find timezone: using UTC
Version:        5.4.220730 [Community build]
GIT rev:        5.4-stable:122a0829afd09ce10a8a9ea4b6e4d1781bd7ca77:20220730
tusc commented 2 years ago

Further troubleshooting, the httpdocs/dist folder in the built deb package is completely empty. I copied the contents of this folder from the official docker x64 image and now the web pages work properly.

Am I missing a step in the build process to populate the dist folder?

MatteoBiscosi commented 2 years ago

Hi @tusc , we started migrating the GUI to a client side app (using Vue). For this reason we did an other repository on github (https://github.com/ntop/ntopng-dist) and here you can find all the css you need and the js, so you need to add this repo into the ntopng folder httpdocs and rename it dist. We automatically do this when running the autogen.sh

matteo@matteo-Precision-3541:~/ntopng/httpdocs$ ls -ln total 52 drwxrwxr-x 2 1000 1000 4096 lug 11 10:54 css drwxrwxr-x 3 1000 1000 4096 lug 27 12:34 dist -rw-rw-r-- 1 1000 1000 1150 feb 14 17:57 favicon.ico drwxrwxr-x 2 1000 1000 4096 lug 11 10:54 geoip drwxrwxr-x 2 1000 1000 4096 lug 11 10:54 img drwxrwxr-x 2 1000 1000 4096 lug 11 10:54 inc drwxrwxr-x 3 1000 1000 4096 lug 27 15:56 misc drwxrwxr-x 2 1000 1000 4096 feb 14 17:57 other drwxrwxr-x 2 1000 1000 4096 feb 14 17:57 ssl drwxrwxr-x 5 1000 1000 4096 lug 11 10:54 templates drwxrwxr-x 2 1000 1000 4096 lug 11 10:54 test drwxrwxr-x 3 1000 1000 4096 lug 11 10:54 tmp drwxrwxr-x 2 1000 1000 4096 lug 12 11:23 tmp-dist

tusc commented 2 years ago

Matteo, thanks!

I believe that is the issue. I did notice during the autogen.sh process there were some failures:

root@a4f091976e7a:~/ntopng# ./autogen.sh
Submodule 'dist' (https://github.com/ntop/ntopng-dist.git) registered for path 'httpdocs/dist'
Submodule 'tests/e2e' (https://github.com/ntop/ntopng-e2e-tests.git) registered for path 'tests/e2e'
Cloning into '/root/ntopng/httpdocs/dist'...
fatal: unable to access 'https://github.com/ntop/ntopng-dist.git/': server certificate verification failed. CAfile: none CRLfile: none
fatal: clone of 'https://github.com/ntop/ntopng-dist.git' into submodule path '/root/ntopng/httpdocs/dist' failed
Failed to clone 'httpdocs/dist'. Retry scheduled
Cloning into '/root/ntopng/tests/e2e'...
fatal: unable to access 'https://github.com/ntop/ntopng-e2e-tests.git/': server certificate verification failed. CAfile: none CRLfile: none
fatal: clone of 'https://github.com/ntop/ntopng-e2e-tests.git' into submodule path '/root/ntopng/tests/e2e' failed
Failed to clone 'tests/e2e'. Retry scheduled
Cloning into '/root/ntopng/httpdocs/dist'...
fatal: unable to access 'https://github.com/ntop/ntopng-dist.git/': server certificate verification failed. CAfile: none CRLfile: none
fatal: clone of 'https://github.com/ntop/ntopng-dist.git' into submodule path '/root/ntopng/httpdocs/dist' failed
Failed to clone 'httpdocs/dist' a second time, aborting
Wait please...

Now run ./configure

It appears I was missing ca-certificates package on the debian build container. After installing this package, the autogen.sh completed without issue. I'm going to close this issue.

root@a4f091976e7a:~/ntopng# ./autogen.sh
Cloning into '/root/ntopng/httpdocs/dist'...
Cloning into '/root/ntopng/tests/e2e'...
Submodule path 'httpdocs/dist': checked out '604aa391f4aebd9edd3e5942ed718bbf90614a1a'
Submodule path 'tests/e2e': checked out '7d6c2607d216d59341f31d80082fd430141107b0'
Wait please...

Now run ./configure