Closed umarizulkifli closed 6 years ago
dalibort
commented
8 years ago Hi,
It looks like netflow_collector.cpp is complaining that he has no definition of the current set of values sent by netflow collector. what device you are using to collect the netflow data? Could you verify (by a tcpdump/wireshark for example) that the flowsets are present in the data? Maybe you could try to use lower netflow version than v9 (one which uses hardcoded data format i.e v5 ) ?
pavel-odintsov
commented
8 years ago Please try to decrease template timeout for netflow.
pavel-odintsov
commented
8 years ago Dalibort, thanks for answering!
On Wednesday, 24 August 2016, Pavel Odintsov pavel.odintsov@gmail.com wrote:
zimage
commented
8 years ago I'm having a similar issue, but it seems to be only..
2016-09-07 21:46:24,281 [INFO] We don't have a template for flowset_id: 512 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it!
2016-09-07 21:46:34,137 [INFO] Received ipfix options flowset id, which is not supportedI'm using Juniper MX104 routers with the exact config you have in your documentation.
pavel-odintsov
commented
8 years ago Could u update to git version? It was fixed some time ago: https://fastnetmon.com/install/.
zimage
commented
8 years ago I deleted everything in /opt that was installed by my initial install. then reinstalled using
$ sudo perl fastnetmon_install.pl --use-git-master … $ /opt/fastnetmon/fastnetmon --version Version: 1.1.3 master git-ab4ce87172f1d92d68941c84661e897385a6d54c
After starting fastnetmon with
$ sudo service fastnetmon start
This is my log file. Still a ton of We don't have a template for flowset_id: 256 and 512
[INFO] Logger initialized! [ERROR] Can't parse config line: 'ban_for_tcp_pps = off ' [INFO] Read configuration file [INFO] We are working on Linux and could use ip tool for detecting local IP's [INFO] We found 1 local IP addresses and will monitor they [INFO] We loaded 20 networks from networks file [INFO] Totally we have 21 IPv4 subnets [INFO] Totally we have 0 IPv6 subnets [INFO] Total number of monitored hosts (total size of all networks): 14595 [INFO] We need 8 MB of memory for storing counters for your networks [INFO] I will allocate 1 records for subnet 1725374258 cidr mask: 32 [INFO] I will allocate 4096 records for subnet 3166016 cidr mask: 20 [INFO] I will allocate 1 records for subnet 405950272 cidr mask: 32 [INFO] I will allocate 256 records for subnet 6998592 cidr mask: 24 [INFO] I will allocate 256 records for subnet 8374848 cidr mask: 24 [INFO] I will allocate 512 records for subnet 13558082 cidr mask: 23 [INFO] I will allocate 256 records for subnet 16507202 cidr mask: 24 [INFO] I will allocate 1024 records for subnet 2128714 cidr mask: 22 [INFO] I will allocate 512 records for subnet 2909771 cidr mask: 23 [INFO] I will allocate 1024 records for subnet 11557831 cidr mask: 22 [INFO] I will allocate 256 records for subnet 16014796 cidr mask: 24 [INFO] I will allocate 512 records for subnet 10283725 cidr mask: 23 [INFO] I will allocate 512 records for subnet 11594445 cidr mask: 23 [INFO] I will allocate 1024 records for subnet 4999376 cidr mask: 22 [INFO] I will allocate 1024 records for subnet 5526224 cidr mask: 22 [INFO] I will allocate 1024 records for subnet 11557328 cidr mask: 22 [INFO] I will allocate 1 records for subnet 1644380888 cidr mask: 32 [INFO] I will allocate 512 records for subnet 1636056 cidr mask: 23 [INFO] I will allocate 1024 records for subnet 4519640 cidr mask: 22 [INFO] I will allocate 256 records for subnet 6354648 cidr mask: 24 [INFO] I will allocate 512 records for subnet 7927512 cidr mask: 23 [INFO] We start total zerofication of counters [INFO] We finished zerofication [INFO] We loaded 21 IPv4 subnets to our in-memory list of networks [INFO] netflow plugin started [INFO] Using custom sampling ratio for netflow: 1 [INFO] netflow: We will listen on 1 ports [INFO] Run banlist cleanup thread, we will awake every 60 seconds [INFO] netflow plugin will listen on 0.0.0.0:2055 udp port [INFO] We don't have a template for flowset_id: 256 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! ... repeated 4520 more times ... [INFO] We don't have a template for flowset_id: 512 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! ... repeated 7 more times before I stopped the service. These were coming in about two every ten seconds …
pavel-odintsov
commented
8 years ago What is your options template timeout at router side? Please increase it up to 10-15 seconds. This record only about inability to read template for data arriving from your router.
zimage
commented
8 years ago On Sep 8, 2016, at 2:08 PM, Pavel Odintsov notifications@github.com wrote:
What is your options template timeout at router side? Please increase it up to 10-15 seconds. This record only about inability to read template for data arriving from your router.
services { flow-monitoring { version-ipfix { template ipv4 { flow-active-timeout 10; flow-inactive-timeout 10; template-refresh-rate { packets 1000; seconds 10; } option-refresh-rate { packets 1000; seconds 10; } ipv4-template; } } } }
I will change the template-refresh-rate to 15 seconds on both peering routers.
Tim
pavel-odintsov
commented
8 years ago 10 seconds is also OK. So it should work...
zimage
commented
8 years ago It was set to 10, but it wasn’t working. I just increased the template-refresh-rate to 15. In a meeting but I’ll check logs later.
pavel-odintsov
commented
8 years ago Have you solved issue?
pavel-odintsov
commented
7 years ago Feel free to reopen this ticket if you still have issues.
Absynth723
commented
7 years ago I'm having the same issue with latest exabgp from git. In my case, I'm using Wanguard's netflow repeater capabilities.
2017-05-23 16:26:18,353 [INFO] We don't have a template for flowset_id: 328 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2017-05-23 16:26:18,356 [INFO] We don't have a template for flowset_id: 328 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2017-05-23 16:26:19,346 [INFO] We don't have a template for flowset_id: 328 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it!
The netflow repeater has no settings for timeouts, I'm afraid. Do you have any idea how to debug/solve the issue?
Absynth723
commented
7 years ago In addition to the log lines before, I regularly get this: 2017-05-23 16:39:38,327 [INFO] I received netflow v9 options flowset id but I haven't support for it
pavel-odintsov
commented
7 years ago Hello!
First advice, do not use middleboxes if you work with NetFlow v9 or IPFIX. It will not work. Ever.
Because FastNetMon uses only one available way to distinguish different templates from different devices (IP address of the device).
But your middlebox (in this case Wanguard) see this information but hides this information from any targets (FastNetMon in this case). And makes impossible to distinguish different devices and corrupt incoming data completely / trigger segmentation faults in the worst case.
Also, middlebox could introduce very strange things sometimes and break protocol implementation at all. We are working very hard to fix all "vendor specific" things and we are working pretty well for all possible cases with almost any vendor.
Finally, please avoid middleboxes and feed data to FastNetMon directly.
Absynth723
commented
7 years ago Thanks for the information - I will test it without the middlebox and feed directly from my routers. It's only a lab environment / Proof of Concept, so that will work.
Absynth723
commented
7 years ago Test with a direct feed was successful. I found another problem that made it seem like no flows were arriving at the fastnetmon instance - the /etc/networks_list file was named /etc/networks.list on my test machine, and thus, only traffic to that machine's local ip address was shown in fastnetmon client. All good now, and it even works with netflow multiplexer from WANguard like a charm, muxing two netflow exporters onto fastnetmon's port 2055.
pavel-odintsov
commented
7 years ago Great news! But anyway, I would recommend avoid middleboxes if you have multiple different vendors/models of NetFlow agentsю
So, I'm closing ticket!
optimuscream
commented
6 years ago Hi,
Pardon me to re-open the old thread, following the issue above . I still have issue with many error from /var/log/fastnetmon.log :
2018-02-06 10:01:37,298 [INFO] We don't have a template for flowset_id: 512 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2018-02-06 10:01:37,299 [INFO] We don't have a template for flowset_id: 513 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it!
Increase the refresh rate from 10 to 15 , still no avail
template ipv4 { flow-active-timeout 10; flow-inactive-timeout 10; template-refresh-rate { packets 1000; seconds 15; } option-refresh-rate { packets 1000; seconds 15; } ipv4-template; } template ipv6 { flow-active-timeout 10; flow-inactive-timeout 10; template-refresh-rate { packets 1000; seconds 15; } option-refresh-rate { packets 1000; seconds 15; } ipv6-template; }
Model: mx480 Junos: 17.3R1.10
I open another ticket if this not allowed.
Thank you.
pavel-odintsov
commented
6 years ago Hello!
Usually, you could ignore such errors. Does it show correct bandwidth in fastnetmon_client? If yes, it’s absolutely safe to ignore them.
Also, you could create pcap file for 5-10 minutes and share with me privately pavel@fastnetmon.com I will check it.
optimuscream
commented
6 years ago Helo Pavel
Thank you for the respons. I'm using samplicator (https://github.com/sleinen/samplicator) to duplicate traffic from real interface to the loopback address of my netflow machine
I can monitor traffic by nfsen (nfsen.sourceforge.net) and also by fastnetmon. Each ot them listen on the different port on loopback IP address. Packet received is "spoofed" so the receiver only now the packet directly come from the network device.
On the second though I think samplicator maybe the culprit here. Give me time to collect data and analysis.
I want to change samplicator to iptables --tee to duplicate the packet and I will share the result.
Thank you.
pavel-odintsov
commented
6 years ago Hello!
Can I ask about devices number in your setup? Do you have more than one router?
optimuscream
commented
6 years ago Hi, Yes, I have : 4 MX4803 Brocade MLX Is that the issue ?What would you recommend ? Thank youRegards Royke
pavel-odintsov
commented
6 years ago Hello!
Yes, you hit a known issue. FastNetMon uses client_ip + source_id from Netflow stream to distinguish different templates.
When you use any kind of intermediate tool you significantly increase the possibility of conflict because you share client_ip for all devices.
The best option to stream traffic directly to FastNetMon. Also, as an option you could change source_id for your devices to distinguish them.
Let me know if you need any help!
pavel-odintsov
commented
6 years ago In response to private email.
Yes, it looks strange. Could you create pcap dump for 5-10 minutes (without nfsen/samplicator, please) and share it with me privately?
Thank you!
pavel-odintsov
commented
6 years ago You provided sFlow dump. It does not have any relations with error discussed in this ticket because it's different protocol and we process it with different engine. If you use any intermediate software like NFSEN please disable it and feed traffic directly.
optimuscream
commented
6 years ago Hello I sent you two pcap file . The em2.pcap is netflow packet . Also I activate sflow = on in fastnetmon.conf configuration I have already try fo feed directly the packet without samplicator and disable nfsen. The result is the same error. I already inform this before.
Anyway ok.Thank you for the time. Regards
On Sunday, 11 February 2018, 23:40, Pavel Odintsov <notifications@github.com> wrote:You provided sFlow dump. It does not have any relations with error discussed in this ticket because it's different protocol and we process it with different engine. If you use any intermediate software like NFSEN please disable it and feed traffic directly.— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
optimuscream
commented
6 years ago Hello
I know my mistake , please apologize The file em1.pcap.gz should be rename em2.pcap.gz or just extract it. Because in it is the file generated from em2 interface which is netflow packet.The file generated is so big so I have to compress it but I forgot to rename it from command line. I usually use the arrow key in my keyboard to track my previous command. Please check if you still want to check. Sorry, my bad :( Thank you. Regards
On Sunday, 11 February 2018, 23:40, Pavel Odintsov <notifications@github.com> wrote:You provided sFlow dump. It does not have any relations with error discussed in this ticket because it's different protocol and we process it with different engine. If you use any intermediate software like NFSEN please disable it and feed traffic directly.— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
pavel-odintsov
commented
6 years ago Hello!
No problem ;) It also my fault. I was sure that I checked both pcap files. Probably, I opened same twice. Will check it soon.
On Mon, 12 Feb 2018 at 07:44, optimuscream notifications@github.com wrote:
Hello
I know my mistake , please apologize The file em1.pcap.gz should be rename em2.pcap.gz or just extract it. Because in it is the file generated from em2 interface which is netflow packet.The file generated is so big so I have to compress it but I forgot to rename it from command line. I usually use the arrow key in my keyboard to track my previous command. Please check if you still want to check. Sorry, my bad :( Thank you. Regards
On Sunday, 11 February 2018, 23:40, Pavel Odintsov < notifications@github.com> wrote:
You provided sFlow dump. It does not have any relations with error discussed in this ticket because it's different protocol and we process it with different engine. If you use any intermediate software like NFSEN please disable it and feed traffic directly.— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pavel-odintsov/fastnetmon/issues/553#issuecomment-364847989, or mute the thread https://github.com/notifications/unsubscribe-auth/ACnfZpxA6XJI30gVZQe7wrMIGLjsEA2cks5tT-vMgaJpZM4JqffT .
-- Sincerely yours, Pavel Odintsov
pavel-odintsov
commented
6 years ago Hello!
I checked both attached files. Both of them has only sFlow traffic (or very-very-very small amount of Netflow).
Please isolate only Netflow traffic and send dump only with Netflow traffic (replace 2055 by your port):
tcpdump -w /root/netflow_data.pcap -n 'udp dst port 2055'Hi Pavel, So sorry for misunderstanding,I don't know it's me or google drive to blame here :) Here is the new attachment from em2 interface on my netflow machine. I bypassed samplicator and use direct traffic captured on the interface by tcpdump on port 9997 originated from router
So I don't use any other option in tcpdump.
tcpdump -i em2 -W 1 -G 300 -w em2.pcap
Thank you.
On Monday, 12 February 2018, 17:16, Pavel Odintsov <notifications@github.com> wrote:Hello!I checked both attached files. Both of them has only sFlow traffic (or very-very-very small amount of Netflow).Please isolate only Netflow traffic and send dump only with Netflow traffic (replace 2055 by your port):tcpdump -w /root/netflow_data.pcap -n 'udp dst port 2055' — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
pavel-odintsov
commented
6 years ago Hello!
I still do not see pcap dumps attached to ticket.
This info is eating up my logfile. How to turn down the logging level
Thank you