Open apezio opened 1 week ago
apezio
commented
1 week ago I looked at the packets exported by nprobe and see a field called "packets" with a value of "32", in each flow. I don't see a "Sampling Interval" (cflow.sampling_interval) field which is what I was trying to check for... Shouldn't nprobe be exporting a sampling interval field with the nflite sampling interval into these v9 exports?
apezio
commented
1 week ago For now I bypassed the problem in nprobe by configuring my nfcapd with '-s 32' since upscaling in nprobe doesnt work for me. Still haven't found a way to get it looking correct in ntopng but this works for nfsen.
lucaderi
commented
5 days ago As of today, nProbe is multiplying bytes and packets for the sampling rate (32 in your case) so the behaviour you report is what I would expect
nprobe traffic is inaccurate when using the nflite plugin. Pretty sure it's off by a factor of 32x, which is nflite's default sample rate (1:32 packets). I am exporting to nfsen (because nflite exporting to ntopng barely works at all).
Collector Threads: [36599 pkts] Processed packets: 1001824 (max bucket search: 0)
These numbers (Processed packets) seem to match what my router reports, but nprobe doesnt actually export these stats to ntopng/nfsen.
Version: 10.5.240618 Build OS: CentOS Linux release 7.9.2009 (Core) SystemID: LA2C62ABCB205A206--UA2C62ABC9F24C390--OL GIT rev: dev:48fea405b3d9c583be5bd2ebdea01883ee496977:20240618 Edition: nProbe Enterprise S
nprobe collects from a cisco 4948e with:
nprobe -n 127.0.0.1:2056 --nflite 2055:1 -i none -S 32:1:1
and sends to nfsen -
nfsen reports 8Mbps but my actual traffic is approx 200Mbps (see below)
I have tried: -S 1:299:1 -S 1000:1:1 -S 1:1:1000 -S 1:1:32 -S 1:100:100 -S 100:1:100
None of these -S options seem to have any effect whatsoever.
Cisco config: netflow-lite exporter test transport udp 2055 template data timeout 60 options sampler-table timeout 60 options interface-table timeout 60 source
destination
!
!
netflow-lite sampler test
packet-rate 32
interface TenGigabitEthernet1/52 no switchport ip address 255.255.255.254
netflow-lite monitor 1
sampler test
exporter test
router#sh int te1/52 | inc minute 5 minute input rate 72871000 bits/sec, 9646 packets/sec 5 minute output rate 121590000 bits/sec, 22680 packets/sec
Nprobe reports ~1k pps but the interface has ~30k pps.
I am really confused by the documentation for -S. Do the nflite UDP packets contain the sample rate? Does nflite plugin multiply by 32 to begin with? If I set the sample rate at 32:1:1 does that mean nprobe will only sample 1 our of 32 UDP packets (1 in 1024 real packets?) or is it just telling nprobe what the actual sample rate is being used?
Any help much appreciated!
nprobe/nfsen/nfcapd output:
[ root ntop ~ ] nprobe -n 127.0.0.1:2056 -b 1 --nflite 2055:1 -i none 19/Jun/2024 02:54:51 [plugin.c:178] No plugins found in ./plugins 19/Jun/2024 02:54:51 [plugin.c:186] Loading 23 plugins [.so] from /usr/lib/nprobe/plugins 19/Jun/2024 02:54:52 [nprobe.c:4978] Exporting flows towards 127.0.0.1:2056 using UDP 19/Jun/2024 02:54:52 [nprobe.c:7965] IMPORTANT: Enabling NflitePlugin will also enable IP address forging, thus 19/Jun/2024 02:54:52 [nprobe.c:7966] IMPORTANT: flows appear as they were sent from the NflitePlugin-enabled switch 19/Jun/2024 02:54:52 [nprobe.c:8106] WARNING: The output interfaceId is set to 0: did you forget to use -Q perhaps ? 19/Jun/2024 02:54:52 [nprobe.c:8109] WARNING: The input interfaceId is set to 0: did you forget to use -u perhaps ? 19/Jun/2024 02:54:52 [nprobe.c:8214] Welcome to nProbe v.10.5.240618 for x86_64-unknown-linux-gnu with native PF_RING acceleration 19/Jun/2024 02:54:52 [nprobe.c:8236] Enterprise S Edition running on CentOS Linux release 7.9.2009 (Core) 19/Jun/2024 02:54:52 [nprobe.c:8244] Current limits [8 ZMQ exporters][8 collector devices] 19/Jun/2024 02:54:52 [nprobe.c:8259] SystemId: LA2C62ABCB205A206--UA2C62ABC9F24C390--OL 19/Jun/2024 02:54:52 [nprobe.c:8352] Sample rate [packet: 1][flow collection/export: 1/1] 19/Jun/2024 02:54:52 [nflitePlugin.c:911] ERROR: Flow collector port 2055/IPv6 already in use ? [Address family not supported by protocol/97]: disabling collection over IPv6 19/Jun/2024 02:54:52 [nflitePlugin.c:935] [NFLite] Listening on port range 2055-2055 (1) 19/Jun/2024 02:54:52 [nprobe.c:10400] Using template %IPV4_SRC_ADDR %IPV4_DST_ADDR %INPUT_SNMP %OUTPUT_SNMP %IN_PKTS %IN_BYTES %FIRST_SWITCHED %LAST_SWITCHED %L4_SRC_PORT %L4_DST_PORT %TCP_FLAGS %PROTOCOL %SRC_TOS %SRC_AS %DST_AS 19/Jun/2024 02:54:52 [nprobe.c:10402] Using NetFlow Packet Payload Len: 1472 19/Jun/2024 02:54:52 [plugin.c:1207] 1 plugin(s) enabled 19/Jun/2024 02:54:52 [nprobe.c:10762] Skipping plugin Netflow-Lite Plugin: no IEs defined 19/Jun/2024 02:54:52 [nprobe.c:10947] Each flow is 71 bytes long 19/Jun/2024 02:54:52 [nprobe.c:10948] The # flows per packet has been set to 19 19/Jun/2024 02:54:52 [nprobe.c:10951] IP TOS is accounted 19/Jun/2024 02:54:52 [nprobe.c:10979] Non IPv4/v6 traffic is discarded according to the template 19/Jun/2024 02:54:52 [util.c:571] Loaded database /usr/share/ntopng/httpdocs/geoip/dbip-asn-lite.mmdb [ip_version: 6] 19/Jun/2024 02:54:52 [util.c:605] Loaded database /usr/share/ntopng/httpdocs/geoip/dbip-country-lite.mmdb [ip_version: 6] 19/Jun/2024 02:54:52 [nprobe.c:11803] Flow export type (-T): unidirectional flows 19/Jun/2024 02:54:52 [nprobe.c:12029] Flows will be exported in NetFlow 9 format 19/Jun/2024 02:54:52 [nprobe.c:12267] Not capturing packet from interface (collector mode) 19/Jun/2024 02:54:52 [plugin.c:1002] Enabling plugin Netflow-Lite Plugin 19/Jun/2024 02:54:52 [export.c:487] Using TLV as serialization format 19/Jun/2024 02:54:52 [nprobe.c:12581] nProbe started successfully 19/Jun/2024 02:54:53 [nprobe.c:4407] --------------------------------- 19/Jun/2024 02:54:53 [nprobe.c:4410] Average traffic: [0.00 pps][All Traffic 0 b/sec][IP Traffic 0 b/sec][ratio -nan] 19/Jun/2024 02:54:53 [nprobe.c:4418] Current traffic: [0.00 pps][0 b/sec] 19/Jun/2024 02:54:53 [nprobe.c:4459] Flows exports (including drops) [0 flows][uptime: 1 sec][avg: 0.0 flows/sec][latest 1 sec avg: 0.0 flows/sec] 19/Jun/2024 02:54:53 [nprobe.c:4468] Flow drops [export queue full: 0] 19/Jun/2024 02:54:53 [nprobe.c:4471] Packet drops [too many flow buckets: 0] 19/Jun/2024 02:54:53 [nprobe.c:4474] Flow Buckets [active: 0][allocated: 0][toBeExported: 0] 19/Jun/2024 02:54:53 [nprobe.c:4478] Export Queue [current: 0][max: 512000][fill level: 0.0%] 19/Jun/2024 02:54:53 [nflitePlugin.c:993] [NFLite] [# Template Pkts Rcvd: 0][# Flows with Unknown Templates: 270] 19/Jun/2024 02:54:53 [nflitePlugin.c:996] [NFLite] [# Templates Defined: 0][# Flows Rcvd: 270][# Data Flows: 0][# Bad Flows: 0] 19/Jun/2024 02:54:53 [nflitePlugin.c:1001] [NFLite] [# Flow Packets Lost: 96323353][Flow Sequence: 5991438-102315057 (96323619)][# Flow Rcvd: 266] 19/Jun/2024 02:54:53 [nprobe.c:4551] Collector Threads: [270 pkts] 19/Jun/2024 02:54:53 [nprobe.c:4227] Processed packets: 0 (max bucket search: 0) 19/Jun/2024 02:54:53 [nprobe.c:4208] Fragment queue length: 0 19/Jun/2024 02:54:53 [nprobe.c:4258] Flow export stats: [0 bytes/0 pkts][0 flows/0 pkts sent] 19/Jun/2024 02:54:53 [nprobe.c:4270] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %] 19/Jun/2024 02:54:53 [nprobe.c:4276] Total flow stats: [0 bytes/0 pkts][0 flows/0 pkts sent] Jun 19 02:55:01 ntop nfcapd[15764]: Ident: 'upstream1' Flows: 8209, Packets: 1116992, Bytes: 868864736, Sequence Errors: 0, Bad Packets: 0 Jun 19 02:55:01 ntop nfcapd[15764]: Total ignored packets: 0 Jun 19 02:55:15 ntop nfsen[15766]: 1 channels/alerts to profile Jun 19 02:55:15 ntop nfsen[15766]: Update profile live in group . Jun 19 02:55:15 ntop nfsen[15766]: Update profile HTTP_TRAFFIC in group group2 Jun 19 02:55:15 ntop nfsen[4865]: Run expire at Wed Jun 19 02:55:00 2024 Jun 19 02:55:15 ntop nfsen[4865]: Expire has 228s in this slot! Jun 19 02:55:15 ntop nfsen[4865]: Expire profile live group . low water mark: 90% Jun 19 02:55:15 ntop nfsen[4866]: Plugin Cycle: Time: 202406190250, Profile: live, Group: ., Module: HostStats, Jun 19 02:55:15 ntop nfsen[4866]: HostStats run: Profilegroup: ., Profile: live, Time: 202406190250 Jun 19 02:55:15 ntop hoststatserv: NEW_DATA received (timeslot: 202406190250). Jun 19 02:55:15 ntop hoststatserv: Processing timeslot 202406190250 ... Jun 19 02:55:15 ntop nfsen[4866]: HostStats: NewData request sent Jun 19 02:55:15 ntop nfsen[4866]: Plugin Cycle: Time: 202406190250, Profile: live, Group: ., Module: flowdoh, Jun 19 02:55:15 ntop nfsen[4865]: End expire at Wed Jun 19 02:55:00 2024 Jun 19 02:55:15 ntop hoststatserv: Processing of timeslot 202406190250 done. 19/Jun/2024 02:55:53 [nprobe.c:4407] --------------------------------- 19/Jun/2024 02:55:53 [nprobe.c:4410] Average traffic: [19.64 K pps][All Traffic 9.04 Mb/sec][IP Traffic 632.76 Kb/sec][ratio 0.07] 19/Jun/2024 02:55:53 [nprobe.c:4418] Current traffic: [16.70 K pps][7.68 Mb/sec] 19/Jun/2024 02:55:53 [nprobe.c:4459] Flows exports (including drops) [0 flows][uptime: 60 sec][avg: 0.0 flows/sec][latest 60 sec avg: 0.0 flows/sec] 19/Jun/2024 02:55:53 [nprobe.c:4468] Flow drops [export queue full: 0] 19/Jun/2024 02:55:53 [nprobe.c:4471] Packet drops [too many flow buckets: 0] 19/Jun/2024 02:55:53 [nprobe.c:4474] Flow Buckets [active: 718][allocated: 718][toBeExported: 0] 19/Jun/2024 02:55:53 [nprobe.c:4478] Export Queue [current: 0][max: 512000][fill level: 0.0%] 19/Jun/2024 02:55:53 [nflitePlugin.c:993] [NFLite] [# Template Pkts Rcvd: 6][# Flows with Unknown Templates: 5180] 19/Jun/2024 02:55:53 [nflitePlugin.c:996] [NFLite] [# Templates Defined: 4][# Flows Rcvd: 36599][# Data Flows: 31307][# Bad Flows: 0] 19/Jun/2024 02:55:53 [nflitePlugin.c:1001] [NFLite] [# Flow Packets Lost: 96323202][Flow Sequence: 5991438-102351235 (96359797)][# Flow Rcvd: 36595] 19/Jun/2024 02:55:53 [nprobe.c:4551] Collector Threads: [36599 pkts] 19/Jun/2024 02:55:53 [nprobe.c:4227] Processed packets: 1001824 (max bucket search: 0) 19/Jun/2024 02:55:53 [nprobe.c:4208] Fragment queue length: 0 19/Jun/2024 02:55:53 [nprobe.c:4258] Flow export stats: [0 bytes/0 pkts][0 flows/0 pkts sent] 19/Jun/2024 02:55:53 [nprobe.c:4270] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %] 19/Jun/2024 02:55:53 [nprobe.c:4276] Total flow stats: [0 bytes/0 pkts][0 flows/0 pkts sent] Jun 19 02:56:04 ntop nfcapd[15764]: Process_v9: New exporter: SysID: 17, Domain: 235, IP: 127.0.0.1 Jun 19 02:56:04 ntop nfcapd[15764]: Process_v9: [235] Add template 257 Jun 19 02:56:04 ntop nfcapd[15764]: Process_v9: [235] Add template 258 19/Jun/2024 02:56:53 [nprobe.c:4407] --------------------------------- 19/Jun/2024 02:56:53 [nprobe.c:4410] Average traffic: [20.14 K pps][All Traffic 9.30 Mb/sec][IP Traffic 662.90 Kb/sec][ratio 0.07] 19/Jun/2024 02:56:53 [nprobe.c:4418] Current traffic: [20.57 K pps][9.51 Mb/sec] 19/Jun/2024 02:56:53 [nprobe.c:4459] Flows exports (including drops) [1 flows][uptime: 60 sec][avg: 0.0 flows/sec][latest 60 sec avg: 0.0 flows/sec] 19/Jun/2024 02:56:53 [nprobe.c:4468] Flow drops [export queue full: 0] 19/Jun/2024 02:56:53 [nprobe.c:4471] Packet drops [too many flow buckets: 0] 19/Jun/2024 02:56:53 [nprobe.c:4474] Flow Buckets [active: 1268][allocated: 1268][toBeExported: 0] 19/Jun/2024 02:56:53 [nprobe.c:4478] Export Queue [current: 0][max: 512000][fill level: 0.0%] 19/Jun/2024 02:56:53 [nflitePlugin.c:993] [NFLite] [# Template Pkts Rcvd: 12][# Flows with Unknown Templates: 5180] 19/Jun/2024 02:56:53 [nflitePlugin.c:996] [NFLite] [# Templates Defined: 4][# Flows Rcvd: 75277][# Data Flows: 69873][# Bad Flows: 0] 19/Jun/2024 02:56:53 [nflitePlugin.c:1001] [NFLite] [# Flow Packets Lost: 96323040][Flow Sequence: 5991438-102389751 (96398313)][# Flow Rcvd: 75273] 19/Jun/2024 02:56:53 [nprobe.c:4551] Collector Threads: [75277 pkts] 19/Jun/2024 02:56:53 [nprobe.c:4227] Processed packets: 2235936 (max bucket search: 1) 19/Jun/2024 02:56:53 [nprobe.c:4208] Fragment queue length: 0 19/Jun/2024 02:56:53 [nprobe.c:4258] Flow export stats: [1664 bytes/32 pkts][1 flows/2 pkts sent] 19/Jun/2024 02:56:53 [nprobe.c:4270] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %] 19/Jun/2024 02:56:53 [nprobe.c:4276] Total flow stats: [1664 bytes/32 pkts][1 flows/2 pkts sent] 19/Jun/2024 02:57:53 [nprobe.c:4407] --------------------------------- 19/Jun/2024 02:57:53 [nprobe.c:4410] Average traffic: [20.08 K pps][All Traffic 9.29 Mb/sec][IP Traffic 622.76 Kb/sec][ratio 0.07] 19/Jun/2024 02:57:53 [nprobe.c:4418] Current traffic: [19.97 K pps][9.27 Mb/sec] 19/Jun/2024 02:57:53 [nprobe.c:4459] Flows exports (including drops) [649 flows][uptime: 60 sec][avg: 10.8 flows/sec][latest 60 sec avg: 10.8 flows/sec] 19/Jun/2024 02:57:53 [nprobe.c:4468] Flow drops [export queue full: 0] 19/Jun/2024 02:57:53 [nprobe.c:4471] Packet drops [too many flow buckets: 0] 19/Jun/2024 02:57:53 [nprobe.c:4474] Flow Buckets [active: 1235][allocated: 1235][toBeExported: 0] 19/Jun/2024 02:57:53 [nprobe.c:4478] Export Queue [current: 0][max: 512000][fill level: 0.0%] 19/Jun/2024 02:57:53 [nflitePlugin.c:993] [NFLite] [# Template Pkts Rcvd: 18][# Flows with Unknown Templates: 5180] 19/Jun/2024 02:57:53 [nflitePlugin.c:996] [NFLite] [# Templates Defined: 4][# Flows Rcvd: 112836][# Data Flows: 107320][# Bad Flows: 0] 19/Jun/2024 02:57:53 [nflitePlugin.c:1001] [NFLite] [# Flow Packets Lost: 96322887][Flow Sequence: 5991438-102427157 (96435719)][# Flow Rcvd: 112832] 19/Jun/2024 02:57:53 [nprobe.c:4551] Collector Threads: [112836 pkts] 19/Jun/2024 02:57:53 [nprobe.c:4227] Processed packets: 3434240 (max bucket search: 2) 19/Jun/2024 02:57:53 [nprobe.c:4208] Fragment queue length: 0 19/Jun/2024 02:57:53 [nprobe.c:4258] Flow export stats: [165221728 bytes/139808 pkts][649 flows/42 pkts sent] 19/Jun/2024 02:57:53 [nprobe.c:4270] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %] 19/Jun/2024 02:57:53 [nprobe.c:4276] Total flow stats: [165221728 bytes/139808 pkts][649 flows/42 pkts sent]