Fastnetmon client showing 0s only

[shagy234]

Hello!

I just discovered this amazing tool and I am trying to set it up in my network.

Problem is I'm getting only 0s on fastnetmon_client. I am using fastnetmon version 1.1.3 installed on Ubuntu.

fastnetmon.log seems fine:

flow@flowanalizer:~$ tail -f /var/log/fastnetmon.log 2019-10-21 09:34:13,239 [INFO] I will allocate 1 records for subnet 2328390856 cidr mask: 32 2019-10-21 09:34:13,239 [INFO] We start total zerofication of counters 2019-10-21 09:34:13,239 [INFO] We finished zerofication 2019-10-21 09:34:13,239 [INFO] We loaded 2 IPv4 subnets to our in-memory list of networks 2019-10-21 09:34:13,240 [INFO] Run banlist cleanup thread, we will awake every 60 seconds 2019-10-21 09:34:13,240 [INFO] netflow plugin started 2019-10-21 09:34:13,240 [INFO] Using custom sampling ratio for netflow: 1 2019-10-21 09:34:13,240 [INFO] netflow: We will listen on 2 ports 2019-10-21 09:34:13,240 [INFO] netflow plugin will listen on 192.168.20.151:9995 udp port 2019-10-21 09:34:13,240 [INFO] netflow plugin will listen on 192.168.20.151:9991 udp port

at the moment i only have one /32 in my networks_list (for testing), fastnetmon_client shows it when i send some traffic but it only shows 0s..

FastNetMon 1.1.3 master git- Pavel Odintsov: stableit.ru IPs ordered by: packets Incoming traffic 0 pps 0 mbps 0 flows XXX.XXX.200.138 0 pps 0 mbps 0 flows

Outgoing traffic 0 pps 0 mbps 0 flows Internal traffic 0 pps 0 mbps

Other traffic 354 pps 2 mbps

Screen updated in: 0 sec 359 microseconds Traffic calculated in: 0 sec 17 microseconds Total amount of IPv6 packets related to our own network: 0 Not processed packets: 0 pps

As my understanding, fastnetmon "sees" the flow packets as i can send them to log using DUMP_ALL_PACKETS=yes ./fastnetmon

2019-10-21 09:30:46,741 [INFO] Dump: 2019-10-21 09:30:48.000000 36.111.191.73:58844 > 190.99.105.64:3389 protocol: tcp flags: psh,ack frag: 0 packets: 1 size: 687 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,742 [INFO] Dump: 2019-10-21 09:30:48.000000 45.57.6.169:443 > 200.108.204.18:50032 protocol: tcp flags: ack frag: 0 packets: 1 size: 1518 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,742 [INFO] Dump: 2019-10-21 09:30:48.000000 172.217.28.164:443 > 200.108.250.249:57779 protocol: tcp flags: psh,ack frag: 0 packets: 1 size: 1048 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,742 [INFO] Dump: 2019-10-21 09:30:48.000000 198.15.107.51:9952 > 200.108.209.119:49561 protocol: tcp flags: ack frag: 0 packets: 2 size: 3036 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,742 [INFO] Dump: 2019-10-21 09:30:48.000000 2.16.188.139:443 > 201.221.6.85:37858 protocol: tcp flags: ack frag: 0 packets: 1 size: 1518 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:47.000000 201.221.6.88:80 > 190.108.19.219:50137 protocol: tcp flags: ack frag: 0 packets: 1 size: 1514 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:46.775106400 198.45.49.135:443 > 200.108.241.103:62563 protocol: tcp flags: ack frag: 0 packets: 1 size: 1382 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:46.775106400 172.217.30.129:443 > 201.221.31.223:51287 protocol: tcp flags: ack frag: 0 packets: 1 size: 1382 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:46.775106400 201.221.6.90:80 > 200.108.250.245:63771 protocol: tcp flags: ack frag: 0 packets: 3 size: 4500 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:46.775106400 45.57.7.139:80 > 200.108.208.167:40255 protocol: tcp flags: ack frag: 0 packets: 26 size: 28349 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:46.775106400 65.52.108.90:443 > 200.108.218.202:49813 protocol: tcp flags: ack frag: 0 packets: 2 size: 92 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:46.775106400 69.164.45.94:443 > 190.108.23.54:64828 protocol: tcp flags: ack frag: 0 packets: 2 size: 2764 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:46.775106400 200.108.192.4:53 > 201.221.31.51:56296 protocol: udp frag: 0 packets: 1 size: 94 bytes ttl: 0 sample ratio: 1

2019-10-21 09:30:46,789 [INFO] Dump: 2019-10-21 09:30:46.775106400 35.186.121.254:443 > 200.108.207.131:56250 protocol: tcp flags: ack frag: 0 packets: 1 size: 52 bytes ttl: 0 sample ratio: 1

My fastnetmon.conf file:

Main configuration params

Logging configuration

logging:local_syslog_logging = off

logging:remote_syslog_logging = off

logging:remote_syslog_server = 10.10.10.10 logging:remote_syslog_port = 514

enable_ban = on

process_incoming_traffic = on process_outgoing_traffic = off

ban_details_records_count = 500

ban_time = 300

unban_only_if_attack_finished = on

enable_subnet_counters = off

networks_list_path = /etc/networks_list

white_list_path = /etc/networks_whitelist

check_period = 1

enable_connection_tracking = off

ban_for_pps = on ban_for_bandwidth = on ban_for_flows = on

Limits for Dos/DDoS attacks

threshold_pps = 20000 threshold_mbps = 2 threshold_flows = 3500

threshold_tcp_mbps = 100000 threshold_udp_mbps = 1 threshold_icmp_mbps = 100000

threshold_tcp_pps = 100000 threshold_udp_pps = 100000 threshold_icmp_pps = 100000

ban_for_tcp_bandwidth = off ban_for_udp_bandwidth = on ban_for_icmp_bandwidth = off

ban_for_tcp_pps = off ban_for_udp_pps = on ban_for_icmp_pps = off

PF_RING traffic capture, fast enough but the wirespeed version needs a paid license

mirror = off

pfring_sampling_ratio = 1

mirror_netmap = off

mirror_snabbswitch = off

mirror_afpacket = off

interfaces_snabbswitch = 0000:04:00.0,0000:04:00.1,0000:03:00.0,0000:03:00.1

Port mirroring sampling ratio

netmap_sampling_ratio = 1

maximum-packet-length 110; netmap_read_packet_length_from_ip_header = off

pcap = off netflow = on sflow = off

enable_pf_ring_zc_mode = off

interfaces = eth3,eth4

average_calculation_time = 5

average_calculation_time_for_subnets = 20

Netflow configuration

it's possible to specify multiple ports here, using commas as delimiter

netflow_port = 9991,9995 netflow_host = 192.168.20.151

Netflow v9 and IPFIX agents use different and very complex approaches for notifying about sample ratio

Here you could specify a sampling ratio for all this agents

For NetFLOW v5 we extract sampling ratio from packets directely and this option not used

netflow_sampling_ratio = 1

netflow_divide_counters_on_interval_length = off

sflow_port = 6343 sflow_host = 0.0.0.0

Actions when attack detected

notify_script_path = /usr/local/bin/notify_about_attack.sh notify_script_pass_details = on

collect_attack_pcap_dumps = off

process_pcap_attack_dumps_with_dpi = off

redis_enabled = off

redis_port = 6379 redis_host = 127.0.0.1

redis_prefix = mydc1

mongodb_enabled = off mongodb_host = localhost mongodb_port = 27017 mongodb_database_name = fastnetmon

pfring_hardware_filters_enabled = off

exabgp = on exabgp_command_pipe = /var/run/exabgp.cmd exabgp_community = 20255:666

exabgp_next_hop = 192.0.2.1

exabgp_announce_host = on

exabgp_announce_whole_subnet = off

exabgp_flow_spec_announces = off

gobgp = off gobgp_next_hop = 0.0.0.0 gobgp_announce_host = on gobgp_announce_whole_subnet = off

graphite = off graphite_host = 127.0.0.1 graphite_port = 2003

graphite_prefix = fastnetmon

monitor_local_ip_addresses = on

hostgroup = my_hosts:10.10.10.221/32,10.10.10.222/32

my_hosts_enable_ban = off

my_hosts_ban_for_pps = off my_hosts_ban_for_bandwidth = off my_hosts_ban_for_flows = off

my_hosts_threshold_pps = 20000 my_hosts_threshold_mbps = 1000 my_hosts_threshold_flows = 3500

pid_path = /var/run/fastnetmon.pid

cli_stats_file_path = /tmp/fastnetmon.dat

sort_parameter = packets max_ips_in_list = 7

Maybe it has something to do with the fact that i am using huawei's netstream?, Router's netstream configuration:

ip netstream export host 200.108.XXX.XXX 9991 ip netstream sampler fix-packets 10000 inbound ip netstream sampler fix-packets 10000 outbound

**Thank you for this amazing tool Pavel Odintsov!

regards,

Ramiro.**

[pavel-odintsov]

Hello!

Thank you for feedback!

Your sampling rate is extremely big. I can suggest starting from 1:1024 and you did not set sampling rate in FastNetMon's configuration: netflow_sampling_ratio = 1

Have you added all your client networks in networks_list?

[shagy234]

Hello Pavel!

I dont know if i can change that value on the router without affecting its resources (CPU ?), its an ISP border router.

I have changed fastnetmon.conf to netflow_sampling_ratio=10000 and it seems to work randomly. Most of time im getting this error in log:

2019-10-21 14:25:54,017 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,026 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,253 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,256 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,270 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,464 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,484 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,556 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,717 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,723 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,947 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,956 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:54,967 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:55,182 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:55,189 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:55,253 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:55,413 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:55,421 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:55,485 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:55,647 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it! 2019-10-21 14:25:55,654 [INFO] We don't have a template for flowset_id: 1315 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it!

I have added only a few networks in networks_list for testing.

Thank you!

Ramiro.

[pavel-odintsov]

Yes, router may be easily killed with incorrect sampling rate configuration. But with 10k you will not receive reliable data. You need at least following numbers for sampling rate to get reliable values: https://1.bp.blogspot.com/_N3xuQCvc1v4/SkUhTKcl9KI/AAAAAAAAACU/4o_4MJ9e7sE/s1600/samplingrates.PNG

You can reduce Template Options timeout to suppress such errors. FastNetMon reports inability to decode your netflow data because it does not receive information about data formats used in your Netflow.

[pavel-odintsov]

I can suggest checking our official guides on https://fastnetmon.com/docs/junos_integration/

[shagy234]

Thank you very much Pavel, i got it working now!

Regards, Ramiro.

[pavel-odintsov]

Great! Thank you for confirming!