search

ntop / ntopng

Web-based Traffic and Security Network Traffic Monitoring
http://www.ntop.org
GNU General Public License v3.0
6.23k stars 654 forks source link

"No Data Available" for "Top Application Traffic Last Day View" #1010

Closed gianogli closed 7 years ago

gianogli commented 7 years ago

ntopng I can't understand why we haven't data for this graph also if ntopng is up and running from 18 hours. Is there a bug in ntopng or is it a problem of our configuration?

We run ntopng command with these parameters:

ntopng -i zc:2@0 -i zc:2@1 -i zc:2@2 -i zc:2@3 -i view:zc:2@0,zc:2@1,zc:2@2,zc:2@3 -H -q -n3 -d /var/tmp/data/ntopng/ -m "NET_1,NET_2,NET_3,NET_4,NET_5,NET_6,NET_7,NET_8,NET_9,NET_10,NET_11,NET_12,NET_13,NET_14" -w3000 -r "127.0.0.1:6379" -g 24,25,26,27,28 -l 1 -x 524288 -X 524288 -B "not (udp and src host FILTERED_HOST and dst port FILTERED_PORT) and not net FILTERED_NET" -S local -F "mysql;localhost;ntopng;flows;MYUSER;MYPASS

We are using: ntopng Pro [Small Business Edition] v.2.5.170301 Any idea? What do you recommend to check in my configuration?

Thanks...

simonemainardi commented 7 years ago

Do you get the data if you select any of the non-view interfaces defined?

gianogli commented 7 years ago

Hi, I've just tried to select a single interface at a time but the result is the same:

ntopng2

as you can see, in this case, I choose the interface zc:2@3 but no data are displayed. Any Idea?

Thnaks...

simonemainardi commented 7 years ago

@gianogli some minor fixes have been introduced after version 2.5.170301, please try to update and re-run using the latest dev version.

If charts are still missing, please chdir to /var/tmp/data/ntopng/{IFID}/rrd (where IFID is the interface id assigned by ntopng to one of your interfaces), and see if there are any rrd files named with L7 Protocols, e.g., 

Simones-MacBook-Pro:rrd simone$ pwd
/var/tmp/ntopng/1/rrd
Simones-MacBook-Pro:rrd simone$ ls -lha
total 20304
drwxrwxrwx  117 nobody  wheel   3.9K Mar  6 19:29 .
drwxrwxrwx   11 root    wheel   374B Feb  8 14:40 ..
drwxrwxrwx    3 nobody  wheel   102B Jan 20 11:20 127
drwxrwxrwx    3 nobody  wheel   102B Jan  5 23:15 192
-rw-rw-rw-    1 nobody  wheel    22K Feb  8 17:55 Amazon.rrd
-rw-rw-rw-    1 nobody  wheel    22K Feb  9 22:28 Apple.rrd
-rw-rw-rw-    1 nobody  wheel    22K Feb  9 22:28 AppleiCloud.rrd
-rw-rw-rw-    1 nobody  wheel    22K Feb  9 17:55 AppleiTunes.rrd
-rw-rw-rw-    1 nobody  wheel    22K Jan 12 14:33 BitTorrent.rrd
-rw-rw-rw-    1 nobody  wheel    22K Feb  7 17:26 Cloudflare.rrd
-rw-rw-rw-    1 nobody  wheel    22K Mar  2 23:41 DHCP.rrd
-rw-rw-rw-    1 nobody  wheel    22K Feb  9 17:55 DNS.rrd
-rw-rw-rw-    1 nobody  wheel    22K Mar  2 23:41 Dropbox.rrd

please, report. Thanks.

simonemainardi commented 7 years ago

@gianogli any news?

gianogli commented 7 years ago

Yes, I've some news. :-) In the last 2 days I reconfigured my server and I updated the ntopng packet to the last version (v.2.5.170308). I think that one of the problem was related to the "Layer-7 Application Timeseries" configuration that was set to "off". Now it's "on" but something doesn't work anyway:

ntopng_new1

As you can see now we have in the graph only the data recorded during the active web session. (As you can see there is also a little problem with the scale factor of last data of the "Last Day View" graph).

This is what happen after a reload of the web page:

ntopng_new3

All the data of the graph are deleted! Why? Is it correct?

However if I open a single interface view I've again the "No Data Available" message:

ntopng_new2

These are the files related to the aggregate interfaces view (7):

# ls -lah 7/rrd/
total 7.5M
drwxrwxrwx  7 nobody nogroup 4.0K Mar  8 14:28 .
drwxrwxrwx  6 root   root    4.0K Mar  8 10:58 ..
drwxrwxrwx 19 nobody nogroup 4.0K Mar  8 14:55 10
drwxrwxrwx  3 nobody nogroup 4.0K Mar  8 11:00 130
drwxrwxrwx  8 nobody nogroup 4.0K Mar  8 15:02 172
drwxrwxrwx  3 nobody nogroup 4.0K Mar  8 11:00 192
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Amazon.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 AppleiCloud.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 AppleiTunes.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Apple.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 BGP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 BitTorrent.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:15 bytes.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 CiscoSkinny.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 CiscoVPN.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Citrix_Online.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Citrix.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Cloudflare.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 CNN.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 COAP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 DCE_RPC.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Deezer.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 DHCP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 DNS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Dropbox.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:15 drops.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 eBay.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 eDonkey.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Facebook.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Flash.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 FTP_CONTROL.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 FTP_DATA.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Github.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Git.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 GMail.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 GoogleHangout.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 GoogleMaps.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Google.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 GRE.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 GTP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 H323.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Hotmail.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 HotspotShield.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 HTTP_Connect.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 HTTPDownload.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 HTTP_Proxy.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 HTTP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 ICMP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 IFLIX.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 IGMP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 IMAP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 IMAPS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Instagram.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 IP_in_IP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 IPsec.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 iQIYI.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 IRC.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 KakaoTalk.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Kerberos.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 LastFM.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 LDAP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 LLMNR.rrd
drwxrwxrwx  2 nobody nogroup 4.0K Mar  8 11:00 localstats
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Lync.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Microsoft.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 MQTT.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 MSN.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 MS_OneDrive.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 MsSQL-TDS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 NetBIOS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 NetFlix.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 NFS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 NTP.rrd
-rw-rw-rw-  1 nobody nogroup 677K Mar  8 16:15 num_devices.rrd
-rw-rw-rw-  1 nobody nogroup 677K Mar  8 16:15 num_flows.rrd
-rw-rw-rw-  1 nobody nogroup 677K Mar  8 16:15 num_hosts.rrd
-rw-rw-rw-  1 nobody nogroup 677K Mar  8 16:15 num_http_hosts.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 OCS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Office365.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 OpenDNS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 OpenVPN.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Oracle.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Oscar.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 OSPF.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:15 packets.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 PcAnywhere.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 POP3.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 POPS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 PPStream.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 PPTP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 QQ.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 QUIC.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Radius.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 RDP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Redis.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 RemoteScan.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 RSYNC.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 RTCP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 RTMP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 RTP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 RTSP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 RX.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 SAP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 sFlow.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 ShoutCast.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Sina(Weibo).rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SIP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Skype.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Slack.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SMB.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SMTP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SMTPS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Snapchat.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SNMP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 SOCKS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Spotify.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SSDP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SSH.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SSL_No_Cert.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 SSL.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Starcraft.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Steam.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 STUN.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Syslog.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:15 tcp_lost.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:15 tcp_ooo.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:15 tcp_retransmissions.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 TeamSpeak.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 TeamViewer.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Telegram.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Telnet.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Teredo.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Tor.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Tuenti.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Twitch.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Twitter.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 UBNTAC2.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 UbuntuONE.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Unencryped_Jabber.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Unknown.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 UPnP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Vevo.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Viber.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 VNC.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 Waze.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Webex.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 WhatsApp.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 WhatsAppVoice.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Whois-DAS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Wikipedia.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 WindowsUpdate.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 14:28 WorldOfWarcraft.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 Yahoo.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 YouTube.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 14:28 ZeroMQ.rrd

These are the files related to the interfaces zc:2@0 (3):

# ls -lah 3/rrd/
total 7.4M
drwxrwxrwx  7 nobody nogroup 4.0K Mar  8 13:49 .
drwxrwxrwx  7 root   root    4.0K Mar  8 10:58 ..
drwxrwxrwx 16 nobody nogroup 4.0K Mar  8 14:14 10
drwxrwxrwx  3 nobody nogroup 4.0K Mar  8 11:01 130
drwxrwxrwx  7 nobody nogroup 4.0K Mar  8 14:26 172
drwxrwxrwx  3 nobody nogroup 4.0K Mar  8 11:01 192
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Amazon.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 AppleiCloud.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 AppleiTunes.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Apple.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 BitTorrent.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:24 bytes.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 CiscoSkinny.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 CiscoVPN.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Citrix_Online.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Citrix.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Cloudflare.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 CNN.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 COAP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 DCE_RPC.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Deezer.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 DHCP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 DNS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Dropbox.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:24 drops.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 eBay.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 eDonkey.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Facebook.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 Flash.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 FTP_CONTROL.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 FTP_DATA.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Github.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 GMail.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 GoogleHangout.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 GoogleMaps.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Google.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 GRE.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 GTP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 H323.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Hotmail.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 HotspotShield.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 HTTP_Connect.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 HTTPDownload.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 HTTP_Proxy.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 HTTP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 ICMP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 IFLIX.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 IGMP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 IMAP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 IMAPS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Instagram.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 IPsec.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 iQIYI.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 IRC.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 KakaoTalk.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 Kerberos.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 LastFM.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 LDAP.rrd
drwxrwxrwx  2 nobody nogroup 4.0K Mar  8 11:01 localstats
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 Lync.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Microsoft.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 MQTT.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 MSN.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 MS_OneDrive.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 MsSQL-TDS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 NetBIOS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 NetFlix.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 NTP.rrd
-rw-rw-rw-  1 nobody nogroup 677K Mar  8 16:24 num_devices.rrd
-rw-rw-rw-  1 nobody nogroup 677K Mar  8 16:24 num_flows.rrd
-rw-rw-rw-  1 nobody nogroup 677K Mar  8 16:24 num_hosts.rrd
-rw-rw-rw-  1 nobody nogroup 677K Mar  8 16:24 num_http_hosts.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Office365.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 OpenDNS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 OpenVPN.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:24 packets.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 PcAnywhere.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 POP3.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 POPS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 PPStream.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 PPTP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 QQ.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 QUIC.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Radius.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 RDP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 Redis.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 RSYNC.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 RTCP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 RTMP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 RTP.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 RTSP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 RX.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 sFlow.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Sina(Weibo).rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SIP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Skype.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Slack.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SMB.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SMTP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SMTPS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Snapchat.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SNMP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Spotify.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SSDP.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SSH.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SSL_No_Cert.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 SSL.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 Starcraft.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Steam.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 STUN.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Syslog.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:24 tcp_lost.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:24 tcp_ooo.rrd
-rw-rw-rw-  1 nobody nogroup 700K Mar  8 16:24 tcp_retransmissions.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 TeamSpeak.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 TeamViewer.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Telegram.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Telnet.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Teredo.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Tor.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Twitch.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Twitter.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 UBNTAC2.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 UbuntuONE.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Unencryped_Jabber.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Unknown.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 Vevo.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Viber.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 VNC.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 Waze.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Webex.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 WhatsApp.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 WhatsAppVoice.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 Whois-DAS.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Wikipedia.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 WindowsUpdate.rrd
-rw-rw-rw-  1 nobody nogroup 4.4K Mar  8 13:49 WorldOfWarcraft.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 Yahoo.rrd
-rw-rw-rw-  1 nobody nogroup 3.6K Mar  8 13:49 YouTube.rrd

Thanks...

simonemainardi commented 7 years ago

so:

All the data of the graph are deleted! Why? Is it correct?

data is not deleted:

However if I open a single interface view I've again the "No Data Available" message:

same as above.

Any chance to have a remote access to the box I just want to make sure everything is consistent. If affirmative, please contact me privately at mainardi ntop org

gianogli commented 7 years ago

Ok, thanks! I'll write you to arrange a remote session.

However... now, after 2 days we have the same issue:

ntopng_new4

Thanks...

simonemainardi commented 7 years ago

We have investigated your issue. The point is that you have an Insane number of local hosts so the periodic ntopng activities --those that collect statistics -- are not able to terminate within their deadlines.

Switching the Layer-7 statistics to off alleviates the problem. Alternatively you can reduce the size of your local networks or consider using RRDCached. Here's an issue similar to yours: https://github.com/ntop/ntopng/issues/716

Maybe one day we will remove rrd and design something more efficient, but we can't estimate when we'll do this switch.

In the meanwhile, we will add an error message to signal deadline misses by periodic scripts so to ease the debug.

emanuele-f commented 7 years ago

Check added in 84b8499a236f369f5c682b50f0055f8a011f09a7

simonemainardi commented 7 years ago

@gianogli you should also consider moving to SSD as I think your disk is toooo slow...

dargelm commented 4 years ago

same situation at me image

Version | 3.8.200120 - Professional Edition Number of hosts: 35 Devices: 25 Flows: 243 image Layer-7 Applications: both

image

emanuele-f commented 4 years ago

The development version of ntopng contains some fixes that probably address the issue. Can you try to update to the dev version? Please follow the instructions for your distro at packages.ntop.org .

dargelm commented 4 years ago

what I did: on Centos7/Nethserver

In order to use this repository do (as root):

cd /etc/yum.repos.d/ wget http://packages.ntop.org/centos/ntop.repo -O ntop.repo ... CentOS 7 rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm ... then do: yum erase zeromq3 (Do this once to make sure zeromq3 is not installed) yum clean all yum update yum install pfring-dkms n2disk nprobe ntopng cento PF_RING is now packaged without ZC drivers so you can optionally install them:

Now the installation hangs image

emanuele-f commented 4 years ago

Please try to first remove ntopng and pf_ring and then install the dev version

dargelm commented 4 years ago

OMG...after many minutes the process ends successfully

image

finally:

yum install pfring-drivers-zc-dkms

dargelm commented 4 years ago

now it works: Version | 3.9.200210 (8920) - Professional Edition

THANK YOU for your support

dargelm commented 4 years ago

the next update fails:

ndpi-3.1.0-2255.x86_64                   ntop
* Mon Nov 19 13:00:00 2018 Alfredo Cardigliano <cardigliano@ntop.org> 2.5
- Initial package version
Skipped (dependency problems):
  ndpi.x86_64 0:3.1.0-2255

[root@nethserver ~]#  echo '{"action":"update","packages":[]}' | /usr/bin/setsid /usr/bin/sudo /usr/libexec/nethserver/api/system-packages/update | jq
{
  "state": "running",
  "steps": -1,
  "event": "Initialization"
}
{
  "state": "running",
  "steps": -1,
  "event": "Resolving RPM dependencies"
}
{
  "state": "running",
  "steps": -1,
  "event": "Downloading Packages"
}
{
  "state": "running",
  "steps": -1,
  "event": "Check Package Signatures"
}
{
  "state": "running",
  "steps": -1,
  "event": "Running Test Transaction"
}
{
  "status": "failed",
  "message": "[YumRPMCheckError] [u'ERROR with transaction check vs depsolve:', 'ndpi = 3.1.0-2251 is needed by (installed) pfring-7.5.0-2836.x86_64']",
  "steps": -1,
  "event": null
}
{
  "id": "1581370347",
  "type": "EventFailed",
  "message": "See /var/log/messages"
}
[root@nethserver ~]#

/var/log/messages":

Feb 10 22:35:30 nethserver nmbd[1964]: [2020/02/10 22:35:30.236550,  0] ../source3/libsmb/nmblib.c:909(send_udp)
Feb 10 22:35:30 nethserver nmbd[1964]:  Packet send failed to 192.168.122.255(137) ERRNO=Operation not permitted
Feb 10 22:35:30 nethserver nmbd[1964]: [2020/02/10 22:35:30.236840,  0] ../source3/nmbd/nmbd_packets.c:180(send_netbios_packet)
Feb 10 22:35:30 nethserver nmbd[1964]:  send_netbios_packet: send_packet() to IP 192.168.122.255 port 137 failed
Feb 10 22:35:30 nethserver nmbd[1964]: [2020/02/10 22:35:30.236959,  0] ../source3/nmbd/nmbd_namequery.c:245(query_name)
Feb 10 22:35:30 nethserver nmbd[1964]:  query_name: Failed to send packet trying to query name WORKGROUP<1d>
emanuele-f commented 4 years ago

Please try to remove nDPI and upgrade ntopng again

dargelm commented 4 years ago

1. removing old installation

# yum remove ndpi

Running transaction
  Erasing    : nethserver-ntopng-2.2.0-1.ns7.noarch                                                                     1/8
warning: file /var/run/ntopng: remove failed: No such file or directory
  Erasing    : ntopng-3.9.200210-8580.x86_64                                                                            2/8
warning: /etc/ntopng/ntopng.conf saved as /etc/ntopng/ntopng.conf.rpmsave
  Erasing    : n2disk-3.5.200210-5181.x86_64
Removed:
  ndpi.x86_64 0:3.1.0-2251

Dependency Removed:
  cento.x86_64 0:1.11.200210-554        n2disk.x86_64 0:3.5.200210-5181       nethserver-ntopng.noarch 0:2.2.0-1.ns7
  nprobe.x86_64 0:8.7.200210-6751       ntopng.x86_64 0:3.9.200210-8580       pfring.x86_64 0:7.5.0-2836
  pfring-dkms.noarch 0:7.5.0-2836

Complete!

2. next installation:

# yum install pfring-dkms n2disk nprobe ntopng cento
No package n2disk available.
No package nprobe available.
No package cento available.
Resolving Dependencies
There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).
The program yum-complete-transaction is found in the yum-utils package.
--> Running transaction check
---> Package ntopng.x86_64 0:3.3.180109-3804 will be installed
--> Processing Dependency: pfring = 7.1.0-1681 for package: ntopng-3.3.180109-3804.x86_64
---> Package pfring-dkms.noarch 0:6.4.1-718 will be installed
--> Running transaction check
---> Package pfring.x86_64 0:7.1.0-1681 will be installed
--> Finished Dependency Resolution

Dependencies Resolved
Downloading packages:
(1/3): pfring-dkms-6.4.1-718.noarch.rpm                                                              |  74 kB  00:00:00
(2/3): pfring-7.1.0-1681.x86_64.rpm                                                                  | 1.2 MB  00:00:01
(3/3): ntopng-3.3.180109-3804.x86_64.rpm                                                             | 6.3 MB  00:00:02
Running transaction
  Installing : pfring-7.1.0-1681.x86_64                                                                                 1/3
Created symlink from /etc/systemd/system/multi-user.target.wants/pf_ring.service to /usr/lib/systemd/system/pf_ring.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/cluster.service to /usr/lib/systemd/system/cluster.service.
  Installing : ntopng-3.3.180109-3804.x86_64                                                                            2/3
Setting up redis auto startup
Note: Forwarding request to 'systemctl enable redis.service'.
Creating link under /usr/local/bin
Created symlink from /etc/systemd/system/multi-user.target.wants/ntopng.service to /usr/lib/systemd/system/ntopng.service.
  Installing : pfring-dkms-6.4.1-718.noarch                                                                             3/3
WARNING: /usr/lib64/dkms/common.postinst does not exist.
Loading new pfring-6.4.1 DKMS files...
Building for 3.10.0-1062.12.1.el7.x86_64
Building initial module for 3.10.0-1062.12.1.el7.x86_64
Done.

pf_ring.ko.xz:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/3.10.0-1062.12.1.el7.x86_64/extra/
Adding any weak-modules
depmod: WARNING: /lib/modules/3.10.0-1062.12.1.el7.x86_64/extra/ixgbevf_zc.ko.xz needs unknown symbol pf_ring_zc_dev_handler
depmod: WARNING: /lib/modules/3.10.0-1062.12.1.el7.x86_64/extra/i40e_zc.ko.xz needs unknown symbol pf_ring_zc_dev_handler
depmod: WARNING: /lib/modules/3.10.0-1062.12.1.el7.x86_64/extra/igb_zc.ko.xz needs unknown symbol pf_ring_zc_dev_handler
depmod: WARNING: /lib/modules/3.10.0-1062.12.1.el7.x86_64/extra/ixgbe_zc.ko.xz needs unknown symbol pf_ring_zc_dev_handler
depmod: WARNING: /lib/modules/3.10.0-1062.12.1.el7.x86_64/extra/e1000e_zc.ko.xz needs unknown symbol pf_ring_zc_dev_handler
depmod: WARNING: /lib/modules/3.10.0-1062.12.1.el7.x86_64/extra/e1000e_zc.ko.xz needs unknown symbol pf_ring_skb_ring_handler

depmod....

DKMS: install completed.
  Verifying  : pfring-7.1.0-1681.x86_64                                                                                 1/3
  Verifying  : ntopng-3.3.180109-3804.x86_64                                                                            2/3
  Verifying  : pfring-dkms-6.4.1-718.noarch                                                                             3/3

Installed:
  ntopng.x86_64 0:3.3.180109-3804                               pfring-dkms.noarch 0:6.4.1-718

Dependency Installed:
  pfring.x86_64 0:7.1.0-1681

Complete!
  1. because errors: next uninstallation 
    
Running transaction  
Installing : pfring-7.1.0-1681.x86_64                                                                                 1/3
Created symlink from /etc/systemd/system/multi-user.target.wants/pf_ring.service to /usr/lib/systemd/system/pf_ring.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/cluster.service to /usr/lib/systemd/system/cluster.service.
Installing : ntopng-3.3.180109-3804.x86_64                                                                            2/3
Setting up redis auto startup
Note: Forwarding request to 'systemctl enable redis.service'.
Creating link under /usr/local/bin
Created symlink from /etc/systemd/system/multi-user.target.wants/ntopng.service to /usr/lib/systemd/system/ntopng.service.
Installing : pfring-dkms-6.4.1-718.noarch                                                                             3/3
WARNING: /usr/lib64/dkms/common.postinst does not exist.
Loading new pfring-6.4.1 DKMS files...
Building for 3.10.0-1062.12.1.el7.x86_64
Building initial module for 3.10.0-1062.12.1.el7.x86_64
Done.

pf_ring.ko.xz: Running module version sanity check.

depmod....

DKMS: install completed. Verifying : pfring-7.1.0-1681.x86_64 1/3 Verifying : ntopng-3.3.180109-3804.x86_64 2/3 Verifying : pfring-dkms-6.4.1-718.noarch 3/3

Installed: ntopng.x86_64 0:3.3.180109-3804 pfring-dkms.noarch 0:6.4.1-718

Dependency Installed: pfring.x86_64 0:7.1.0-1681

Complete!


4. `yum-complete-transaction` installs pfring-dkms-6.4.1-178.noarch again
5. again: ` yum remove pfring-dkms`
**6. 3rd Installation:**
`# yum install pfring-dkms n2disk nprobe ntopng cento`

No package n2disk available. No package nprobe available. No package cento available. Resolving Dependencies --> Running transaction check ---> Package ntopng.x86_64 0:3.3.180109-3804 will be installed ---> Package pfring-dkms.noarch 0:6.4.1-718 will be installed --> Finished Dependency Resolution Installed: ntopng.x86_64 0:3.3.180109-3804 pfring-dkms.noarch 0:6.4.1-718

Complete!


This installation is really different from my [first one](https://github.com/ntop/ntopng/issues/1010#issuecomment-584073801)

Installed  Version: 3.3.180109 
upgrade impossible 
couldn't understand this all
dargelm commented 4 years ago

I installed new from version 2.2. I upgrade to nightly build

all is fine