search

ntop / ntopng

Web-based Traffic and Security Network Traffic Monitoring
http://www.ntop.org
GNU General Public License v3.0
6.22k stars 652 forks source link

No graph for snmp host, although it is a switch with plenty of traffic #2619

Closed StewAlexander-com closed 5 years ago

StewAlexander-com commented 5 years ago

I should see a lot of traffic for a router, but instead it shows nothing

image

emanuele-f commented 5 years ago

Please check out sudo journalctl -u ntopng -f while you reload the ntopng page, can you see errors in the log?

StewAlexander-com commented 5 years ago

I keep seeing this:

May 29 13:17:00 ntopng ntopng[9448]: [NetworkInterface.cpp:2011] WARNING: If you have TSO/GRO enabled, please disable it May 29 13:17:00 ntopng ntopng[9448]: 29/May/2019 13:17:00 [NetworkInterface.cpp:2011] WARNING: If you have TSO/GRO enabled, please disable it May 29 13:17:00 ntopng ntopng[9448]: [NetworkInterface.cpp:2013] WARNING: Use sudo ethtool -K eth0 gro off gso off tso off May 29 13:17:00 ntopng ntopng[9448]: 29/May/2019 13:17:00 [NetworkInterface.cpp:2013] WARNING: Use sudo ethtool -K eth0 gro off gso off tso off

Even after I use the sudo ethtool command

StewAlexander-com commented 5 years ago

I have used the sudo ethtool command over a dozen times, appears to do exactly nothing, because the messages come back.

I've tried stopping ntopng first, then issuing the command, does nothing.

ntopng is running on a ProxMox KVM

Network Driver details:

support@ntopng:~$ ethtool -i eth0
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:00:12.0
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

Operating System details:

support@ntopng:~$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"

Netstat statistics

support@ntopng:~$ netstat -s
Ip:
    Forwarding: 1
    3976426850 total packets received
    240 with invalid addresses
    0 forwarded
    0 incoming packets discarded
    3976346620 incoming packets delivered
    3264638657 requests sent out
    8 dropped because of missing route
    10456 fragments received ok
    20912 fragments created
Icmp:
    90975 ICMP messages received
    127 input ICMP message failed
    ICMP input histogram:
        destination unreachable: 70031
        timeout in transit: 3
        echo requests: 20941
    8967801 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 8946860
        echo replies: 20941
IcmpMsg:
        InType3: 70031
        InType8: 20941
        InType11: 3
        OutType0: 20941
        OutType3: 8946860
Tcp:
    12711887 active connection openings
    780813 passive connection openings
    12649296 failed connection attempts
    5406 connection resets received
    6 connections established
    2181655445 segments received
    2182344894 segments sent out
    864 segments retransmitted
    0 bad segments received
    12662320 resets sent
Udp:
    1153735071 packets received
    640244560 packets to unknown port received
    201895 packet receive errors
    1073328528 packets sent
    201895 receive buffer errors
    0 send buffer errors
    IgnoredMulti: 382681
UdpLite:
TcpExt:
    1 resets received for embryonic SYN_RECV sockets
    835395 TCP sockets finished time wait in fast timer
    99 packetes rejected in established connections because of timestamp
    3950798 delayed acks sent
    1406 delayed acks further delayed because of locked socket
    Quick ack mode was activated 218 times
    1303750916 packet headers predicted
    2603503 acknowledgments not containing data payload received
    1303026451 predicted acknowledgments
    TCPSackRecovery: 252
    Detected reordering 133 times using SACK
    4 congestion windows fully recovered without slow start
    TCPDSACKUndo: 183
    3 congestion windows recovered without slow start after partial ack
    378 fast retransmits
    TCPTimeouts: 179
    TCPLossProbes: 1021
    TCPDSACKOldSent: 218
    TCPDSACKRecv: 444
    TCPDSACKOfoRecv: 1
    178 connections reset due to unexpected data
    4427 connections reset due to early user close
    1 connections aborted due to timeout
    TCPDSACKIgnoredNoUndo: 186
    TCPSackShifted: 175
    TCPSackMerged: 679
    TCPSackShiftFallback: 633
    IPReversePathFilter: 99
    TCPRcvCoalesce: 573520
    TCPOFOQueue: 205
    TCPChallengeACK: 15
    TCPSpuriousRtxHostQueues: 45
    TCPAutoCorking: 10740
    TCPWantZeroWindowAdv: 26
    TCPSynRetrans: 209
    TCPOrigDataSent: 2131658760
    TCPHystartTrainDetect: 543
    TCPHystartTrainCwnd: 9699
    TCPACKSkippedTimeWait: 3
    TCPACKSkippedChallenge: 97
    TCPKeepAlive: 12511
IpExt:
    InNoRoutes: 7
    InMcastPkts: 16527
    OutMcastPkts: 358
    InBcastPkts: 382681
    InOctets: 561414611110
    OutOctets: 364278974843
    InMcastOctets: 594972
    OutMcastOctets: 58112
    InBcastOctets: 43544441
    InNoECTPkts: 3976426853
Sctp:
    0 Current Associations
    0 Active Associations
    0 Passive Associations
    0 Number of Aborteds
    0 Number of Graceful Terminations
    0 Number of Out of Blue packets
    0 Number of Packets with invalid Checksum
    0 Number of control chunks sent
    0 Number of ordered chunks sent
    0 Number of Unordered chunks sent
    0 Number of control chunks received
    0 Number of ordered chunks received
    0 Number of Unordered chunks received
    0 Number of messages fragmented
    0 Number of messages reassembled
    0 Number of SCTP packets sent
    0 Number of SCTP packets received
StewAlexander-com commented 5 years ago

Do you happen to know how to turn off the requested networking stuff on a KVM?

emanuele-f commented 5 years ago

I've reopened the previous issue for the kvm stuff. Regarding this issue, is the realtime traffic of the host correct?

StewAlexander-com commented 5 years ago

yes

emanuele-f commented 5 years ago

Can you perform again the steps described in https://www.ntop.org/guides/ntopng/faq.html#permission-denied-errors, wait 10 minutes, and see if the chart is populated?

StewAlexander-com commented 5 years ago

Tried this twice, no change in results ...

emanuele-f commented 5 years ago

Can you provide remote assistance access on email?

simonemainardi commented 5 years ago

please, attache the full ntopng configuration used and the exact page you visited to obtain the screenshot in the first post.

The WARNING: Use sudo ethtool -K eth0 gro off gso off tso off won't affect missing data

StewAlexander-com commented 5 years ago

Sure first the configuration file:

#         The  configuration  file is similar to the command line, with the exception that an equal
#        sign '=' must be used between key and value. Example:  -i=p1p2  or  --interface=p1p2  For
#        options with no value (e.g. -v) the equal is also necessary. Example: "-v=" must be used.
#
#
#       -G|--pid-path
#        Specifies the path where the PID (process ID) is saved. This option is ignored when
#        ntopng is controlled with systemd (e.g., service ntopng start).
#
-G=/var/run/ntopng.pid
#
#       -e|--daemon
#        This  parameter  causes ntop to become a daemon, i.e. a task which runs in the background
#        without connection to a specific terminal. To use ntop other than as a casual  monitoring
#        tool, you probably will want to use this option. This option is ignored when ntopng is
#        controlled with systemd (e.g., service ntopng start)
#
# -e=
#
#       -i|--interface
#        Specifies  the  network  interface or collector endpoint to be used by ntopng for network
#        monitoring. On Unix you can specify both the interface name  (e.g.  lo)  or  the  numeric
#        interface id as shown by ntopng -h. On Windows you must use the interface number instead.
#        Note that you can specify -i multiple times in order to instruct ntopng to create  multi-
#        ple interfaces.
#
  -i=eth0
# -i=eth2
--interface="tcp://127.0.0.1:1235"
#
#       -w|--http-port
#        Sets the HTTP port of the embedded web server.
#
# -w=3000
#
#       -m|--local-networks
#        ntopng determines the ip addresses and netmasks for each active interface. Any traffic on
#        those  networks  is considered local. This parameter allows the user to define additional
#        networks and subnetworks whose traffic is also considered local in  ntopng  reports.  All
#        other hosts are considered remote. If not specified the default is set to 192.168.1.0/24.
#
#        Commas  separate  multiple  network  values.  Both netmask and CIDR notation may be used,
#        even mixed together, for instance "131.114.21.0/24,10.0.0.0/255.0.0.0".
#
  -m=172.16.0.0/16,10.2.0.0/16,10.10.13.0/24,10.6.0.0/16,10.62.0.0/16,10.63.0.0/16,172.65.65.0/24,10.90.0.0/16,172.17.50.0/24,172.17.70.0/24,172.17.80.0/24,10.110.0.0/16,192.168.138.0/24,10.20.20.62/26,172.17.30.0/24,172.17.40.0/24,10.210.0.0/16,172.17.10.0/24,172.17.20.0/24,172.18.40.0/254,172.17.200.0/24,10.255.0.0/24
#
#       -n|--dns-mode
#        Sets the DNS address resolution mode: 0 - Decode DNS responses  and  resolve  only  local
#        (-m)  numeric  IPs  1  -  Decode DNS responses and resolve all numeric IPs 2 - Decode DNS
#        responses and don't resolve numeric IPs 3 - Don't decode DNS responses and don't  resolve
#
  -n=1
#
#       -S|--sticky-hosts
#        ntopng  periodically purges idle hosts. With this option you can modify this behaviour by
#        telling ntopng not to purge the hosts specified by -S. This parameter requires  an  argu-
#        ment  that  can  be  "all"  (Keep  all hosts in memory), "local" (Keep only local hosts),
#        "remote" (Keep only remote hosts), "none" (Flush hosts when idle).
#
  -S=local
#
#       -d|--data-dir
#        Specifies the data directory (it must be writable by the user that is executing ntopng).
#
 -d=/var/lib/ntopng
#
#       -q|--disable-autologout
#        Sets the DNS address resolution mode: 0 - Decode DNS responses  and  resolve  only  local
#        (-m)  numeric  IPs  1  -  Decode DNS responses and resolve all numeric IPs 2 - Decode DNS
#        responses and don't resolve numeric IPs 3 - Don't decode DNS responses and don't  resolve
#
  -n=1
#
#       -S|--sticky-hosts
#        ntopng  periodically purges idle hosts. With this option you can modify this behaviour by
#        telling ntopng not to purge the hosts specified by -S. This parameter requires  an  argu-
#        ment  that  can  be  "all"  (Keep  all hosts in memory), "local" (Keep only local hosts),
#        "remote" (Keep only remote hosts), "none" (Flush hosts when idle).
#
  -S=local
#
#       -d|--data-dir
#        Specifies the data directory (it must be writable by the user that is executing ntopng).
#
 -d=/var/lib/ntopng
#
#       -q|--disable-autologout
#        Disable web interface logout for inactivity.
#
# -q=

How to get to the place where I see no graph:

image image image image image

I do not see any javascript errors in the Chrome Inspector Console

StewAlexander-com commented 5 years ago

If you still would like remote access to my ntopng web instance, please let me know and I will provide over email

StewAlexander-com commented 5 years ago

So this is the ntopng server, and it should be seeing a lot of traffic, instead the graph shows none

image

StewAlexander-com commented 5 years ago

No graphs for....

StewAlexander-com commented 5 years ago

Top Receiver / Top Sender graphing seems to be unavailable no matter what interface or what object I choose, could this be a configuration issue?

StewAlexander-com commented 5 years ago

Per @emanuele-f 

It should be the interface from nProbe, please check out https://www.ntop.org/nprobe/network-monitoring-101-a-beginners-guide-to-understanding-ntop-tools on how to properly setup ntopng and nprobe for communication. If you still have problems, then post the ntopng and nprobe configuration used

I followed this to the best of my knowledge, the configuration for ntopng.conf is in an earlier comment, here is the configuration for nprobe:

support@ntopng:/etc/nprobe$ sudo cat nprobe.conf
[sudo] password for support:
#         The  configuration  file is similar to the command line, with the exception that an equal
#        sign '=' must be used between key and value. Example:  -i=p1p2  or  --interface=p1p2  For
#        options with no value (e.g. -v) the equal is also necessary. Example: "-v=" must be used.
#
#
#       -g|--pid-file
#        Specifies the path where the PID (process ID) is saved. This option is ignored when
#        nProbe is controlled with systemd (e.g., service nProbe start).
#
# -G=/var/run/nprobe.pid
#
#       -G|--daemon-mode
#        This parameter causes nProbe to become a daemon, i.e. a task which runs in background
#        without connection to a specific terminal. To use nProbe other than as a casual monitoring
#        tool, you probably will want to use this option. This option is ignored when nProbe is
#        controlled with systemd (e.g., service nProbe start)
#
# -G=
#
#       -i|--interface
#        Specifies the physical network interface that nProbe will use to perform the
#        monitoring. On Unix you can specify the interface name  (e.g. -i lo) whereas on Windows
#        you must use the interface number instead (see -h to see the list of numeric ids).
#        To disable monitoring from physical interfaces (e.g., when nProbe is used in
#        collector-only mode) specify -i=none
#
# -i=none
# -i=eth1
-i=lo
#
#       -n|--collector
#        Specifies the NetFlow collector that will be used by nProbe to send the monitored
#        flows. This option can be specified multiple times to deliver monitored flows to
#        multiple collectors in round-robin mode. To disable flow export to NetFlow collectors
#        specify -n=none
#
# -n=10.0.0.1:2055
-n=none
#
#       -3|--collector-port
#       Specifies the port that is being used by a NetFlow exporter to send NetFlow to nProbe.
#       Multiple NetFlow exporters can symultaneously send data to nProbe using the same port.
#       In case no NetFlow exporter is sending data it is safe to skip this option.
#
# -3=6363
support@ntopng:/etc/nprobe$
emanuele-f commented 5 years ago

Your nprobe is capturing the loopback interface traffic (lo), this is probably not what you want. Moreover, the --zmq on nprobe as explained in https://www.ntop.org/nprobe/network-monitoring-101-a-beginners-guide-to-understanding-ntop-tools

StewAlexander-com commented 5 years ago

Since the nprobe collector is on the same server as the ntopng server, if the ntopng server is listening on eth0 should nprobe be listening on eth0 also? The KVM running this only has one interface

This server can see all the vlans in our network

StewAlexander-com commented 5 years ago

getting these messages

support@ntopng /e/nprobe> sudo systemctl status nprobe
● nprobe.service - nprobe extensible NetFlow v5/v9/IPFIX probe/collector for IPv4/v6
   Loaded: loaded (/etc/systemd/system/nprobe.service; disabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-06-05 17:03:26 EDT; 1min 22s ago
  Process: 12942 ExecStartPost=/bin/sh -c /bin/echo "$(/bin/date) nprobe StartPost" >> /var/log/ntop-systemd.log (code=exited, status=0/SUCCESS)
  Process: 12937 ExecStartPre=/bin/sh -c /bin/sed "/-g.*$\|-G.*\|--daemon-mode.*\|--pid-file.*/s/^/#/" /etc/nprobe/nprobe.conf > /run/nprobe.conf (cod
  Process: 12926 ExecStartPre=/bin/sh -c /bin/echo "$(/bin/date) nprobe StartPre" >> /var/log/ntop-systemd.log (code=exited, status=0/SUCCESS)
 Main PID: 12941 (nprobe)
    Tasks: 7 (limit: 4915)
   CGroup: /system.slice/nprobe.service
           └─12941 /usr/local/bin/nprobe /run/nprobe.conf

Jun 05 17:03:26 ntopng nprobe[12941]: 05/Jun/2019 17:03:26 [pro/pf_ring.c:373] Initializing PF_RING socket on device lo..
Jun 05 17:03:26 ntopng nprobe[12941]: 05/Jun/2019 17:03:26 [pro/pf_ring.c:415] Dumping traffic statistics on /proc/net/pf_ring/stats/12941-lo.9
Jun 05 17:03:26 ntopng nprobe[12941]: 05/Jun/2019 17:03:26 [pro/pf_ring.c:486] PF_RING enabled on lo
Jun 05 17:03:26 ntopng nprobe[12941]: 05/Jun/2019 17:03:26 [util.c:4721] Initializing ZMQ as server
Jun 05 17:03:26 ntopng nprobe[12941]: 05/Jun/2019 17:03:26 [util.c:4764] Succesfully created ZMQ endpoint tcp://127.0.0.1:1235
Jun 05 17:03:26 ntopng nprobe[12941]: 05/Jun/2019 17:03:26 [util.c:3790] nProbe changed user to 'nprobe'
Jun 05 17:03:26 ntopng nprobe[12941]: error reading link speed on lo
Jun 05 17:03:26 ntopng nprobe[12941]: 05/Jun/2019 17:03:26 [nprobe.c:9582] nProbe started successfully
Jun 05 17:04:03 ntopng nprobe[12941]: 05/Jun/2019 17:04:03 [pro/pf_ring.c:286] WARNING: Invalid packet length: [len=16438][caplen=128]
Jun 05 17:04:03 ntopng nprobe[12941]: 05/Jun/2019 17:04:03 [pro/pf_ring.c:287] WARNING: Please disable LRO/GRO on your NIC (ethtool -K <NIC> gro off g
lines 1-21/21 (END)

after updating the nprobe.conf file with the following

support@ntopng /e/nprobe> cat nprobe.conf
#         The  configuration  file is similar to the command line, with the exception that an equal
#        sign '=' must be used between key and value. Example:  -i=p1p2  or  --interface=p1p2  For
#        options with no value (e.g. -v) the equal is also necessary. Example: "-v=" must be used.
#
#
#       -g|--pid-file
#        Specifies the path where the PID (process ID) is saved. This option is ignored when
#        nProbe is controlled with systemd (e.g., service nProbe start).
#
# -G=/var/run/nprobe.pid
#
#       -G|--daemon-mode
#        This parameter causes nProbe to become a daemon, i.e. a task which runs in background
#        without connection to a specific terminal. To use nProbe other than as a casual monitoring
#        tool, you probably will want to use this option. This option is ignored when nProbe is
#        controlled with systemd (e.g., service nProbe start)
#
# -G=
#
#       -i|--interface
#        Specifies the physical network interface that nProbe will use to perform the
#        monitoring. On Unix you can specify the interface name  (e.g. -i lo) whereas on Windows
#        you must use the interface number instead (see -h to see the list of numeric ids).
#        To disable monitoring from physical interfaces (e.g., when nProbe is used in
#        collector-only mode) specify -i=none
#
# -i=none
# -i=eth1
-i=lo
#
#       -n|--collector
#        Specifies the NetFlow collector that will be used by nProbe to send the monitored
#        flows. This option can be specified multiple times to deliver monitored flows to
#        multiple collectors in round-robin mode. To disable flow export to NetFlow collectors
#        specify -n=none
#
# -n=10.0.0.1:2055
-n=none
--zmq "tcp://127.0.0.1:1235"
#       -3|--collector-port
#       Specifies the port that is being used by a NetFlow exporter to send NetFlow to nProbe.
#       Multiple NetFlow exporters can symultaneously send data to nProbe using the same port.
#       In case no NetFlow exporter is sending data it is safe to skip this option.
#
# -3=6363
support@ntopng /e/nprobe>
emanuele-f commented 5 years ago

You don't need nprobe in order to monitor the interface traffic, you already have ntopng. Please read the online guides for basic understanding of our tools.

StewAlexander-com commented 5 years ago

Looks like the issue is with my understanding of your tool, which is fine as I am new to it, will read up on docs and perhaps do some retooling.

Either way, appreciate your support with this ticket, please close.