netsampler / goflow2

High performance sFlow/IPFIX/NetFlow Collector
BSD 3-Clause "New" or "Revised" License
465 stars 110 forks source link

Missing sflow samples in the output #292

Closed ACodingfreak closed 6 months ago

ACodingfreak commented 7 months ago

Describe the bug

I am using goflow2 from a docker container with a sample rate of 5s on the router for sflow. As shown in below logs, I can see missing sequence numbers

46758 46760 46764 46766 46767 46769 46770 46771

How can i check what happened to remaining sflow packets ?

312:~/goflow2/goflow2$ sudo docker container run -p 6343:6343/udp -p 2055:2055/udp -it --name goflow2 netsampler/goflow2:ee095a9
~ $ ./goflow2  -format=text
INFO[0000] starting GoFlow2                             
INFO[0000] starting collection                           blocking=false count=1 hostname= port=6343 queue_size=1000000 scheme=sflow workers=2
INFO[0000] starting collection                           blocking=false count=1 hostname= port=2055 queue_size=1000000 scheme=netflow workers=2
type=SFLOW_5 time_received_ns=1709064876419634062 sequence_num=46758 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064876419634062 time_flow_end_ns=1709064876419634062 bytes=118 packets=1 src_addr=10.10.11.54 dst_addr=10.10.11.38 etype=IPv4 proto=TCP src_port=57444 dst_port=16443 in_if=5031 out_if=0 src_mac=00:16:3e:c9:e3:eb dst_mac=3c:ec:ef:fc:f7:38 src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=24 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=63335 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064881439694564 sequence_num=46760 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064881439694564 time_flow_end_ns=1709064881439694564 bytes=70 packets=1 src_addr=10.10.11.38 dst_addr=10.10.11.54 etype=IPv4 proto=TCP src_port=16443 dst_port=57444 in_if=0 out_if=5031 src_mac=3c:ec:ef:fc:f7:38 dst_mac=00:16:3e:c9:e3:eb src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=16 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=2218 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064896499476169 sequence_num=46764 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064896499476169 time_flow_end_ns=1709064896499476169 bytes=1518 packets=1 src_addr=10.10.11.38 dst_addr=10.10.11.54 etype=IPv4 proto=TCP src_port=16443 dst_port=57444 in_if=0 out_if=5031 src_mac=3c:ec:ef:fc:f7:38 dst_mac=00:16:3e:c9:e3:eb src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=16 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=3786 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064900143575705 sequence_num=46766 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064900143575705 time_flow_end_ns=1709064900143575705 bytes=450 packets=1 src_addr=10.10.11.38 dst_addr=10.10.11.54 etype=IPv4 proto=TCP src_port=52557 dst_port=17070 in_if=0 out_if=5031 src_mac=3c:ec:ef:fc:f7:38 dst_mac=00:16:3e:c9:e3:eb src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=63 ip_flags=2 tcp_flags=24 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=43298 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064901519248839 sequence_num=46767 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064901519248839 time_flow_end_ns=1709064901519248839 bytes=70 packets=1 src_addr=10.10.11.38 dst_addr=10.10.11.54 etype=IPv4 proto=TCP src_port=16443 dst_port=57444 in_if=0 out_if=5031 src_mac=3c:ec:ef:fc:f7:38 dst_mac=00:16:3e:c9:e3:eb src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=16 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=4193 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064902810200269 sequence_num=46769 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064902810200269 time_flow_end_ns=1709064902810200269 bytes=118 packets=1 src_addr=10.10.11.54 dst_addr=10.10.11.38 etype=IPv4 proto=TCP src_port=57444 dst_port=16443 in_if=5031 out_if=0 src_mac=00:16:3e:c9:e3:eb dst_mac=3c:ec:ef:fc:f7:38 src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=24 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=64231 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064905154028620 sequence_num=46770 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064905154028620 time_flow_end_ns=1709064905154028620 bytes=1292 packets=1 src_addr=10.10.11.38 dst_addr=10.10.11.54 etype=IPv4 proto=TCP src_port=16443 dst_port=57444 in_if=0 out_if=5031 src_mac=3c:ec:ef:fc:f7:38 dst_mac=00:16:3e:c9:e3:eb src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=24 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=4417 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064906539265027 sequence_num=46771 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064906539265027 time_flow_end_ns=1709064906539265027 bytes=1518 packets=1 src_addr=10.10.11.38 dst_addr=10.10.11.54 etype=IPv4 proto=TCP src_port=16443 dst_port=57444 in_if=0 out_if=5031 src_mac=3c:ec:ef:fc:f7:38 dst_mac=00:16:3e:c9:e3:eb src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=16 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=4683 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064926619145851 sequence_num=46776 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064926619145851 time_flow_end_ns=1709064926619145851 bytes=1518 packets=1 src_addr=10.10.11.38 dst_addr=10.10.11.54 etype=IPv4 proto=TCP src_port=16443 dst_port=57444 in_if=0 out_if=5031 src_mac=3c:ec:ef:fc:f7:38 dst_mac=00:16:3e:c9:e3:eb src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=16 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=6323 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0
type=SFLOW_5 time_received_ns=1709064936658975276 sequence_num=46779 sampling_rate=1024 sampler_address=10.10.26.5 time_flow_start_ns=1709064936658975276 time_flow_end_ns=1709064936658975276 bytes=70 packets=1 src_addr=10.10.11.1 dst_addr=10.10.11.62 etype=IPv4 proto=TCP src_port=51616 dst_port=22 in_if=0 out_if=5031 src_mac=52:54:00:f2:63:c5 dst_mac=00:16:3e:e0:03:8b src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=64 ip_flags=2 tcp_flags=16 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=63359 fragment_offset=0 src_as=0 dst_as=0 next_hop= next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop= bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=0 observation_point_id=0

Expected behavior All sflow packet should be processed

ACodingfreak commented 7 months ago

I am attaching sflow capture and respective goflow2 output

Sflow.zip

{"type":"SFLOW_5","time_received_ns":1709065920620635488,"sequence_num":47142,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065920620635488,"time_flow_end_ns":1709065920620635488,"bytes":1518,"packets":1,"src_addr":"10.10.11.38","dst_addr":"10.10.11.54","etype":"IPv4","proto":"TCP","src_port":16443,"dst_port":57444,"in_if":0,"out_if":5031,"src_mac":"3c:ec:ef:fc:f7:38","dst_mac":"00:16:3e:c9:e3:eb","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":25350,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065925640583812,"sequence_num":47144,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065925640583812,"time_flow_end_ns":1709065925640583812,"bytes":70,"packets":1,"src_addr":"10.10.11.1","dst_addr":"10.10.11.62","etype":"IPv4","proto":"TCP","src_port":51616,"dst_port":22,"in_if":0,"out_if":5031,"src_mac":"52:54:00:f2:63:c5","dst_mac":"00:16:3e:e0:03:8b","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":5756,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065930421373036,"sequence_num":47146,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065930421373036,"time_flow_end_ns":1709065930421373036,"bytes":70,"packets":1,"src_addr":"10.10.11.54","dst_addr":"10.10.11.38","etype":"IPv4","proto":"TCP","src_port":57444,"dst_port":16443,"in_if":5031,"out_if":0,"src_mac":"00:16:3e:c9:e3:eb","dst_mac":"3c:ec:ef:fc:f7:38","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":34691,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065930660545905,"sequence_num":47147,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065930660545905,"time_flow_end_ns":1709065930660545905,"bytes":70,"packets":1,"src_addr":"10.10.11.1","dst_addr":"10.10.11.62","etype":"IPv4","proto":"TCP","src_port":51616,"dst_port":22,"in_if":0,"out_if":5031,"src_mac":"52:54:00:f2:63:c5","dst_mac":"00:16:3e:e0:03:8b","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":5842,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065935680520719,"sequence_num":47149,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065935680520719,"time_flow_end_ns":1709065935680520719,"bytes":1518,"packets":1,"src_addr":"10.10.11.38","dst_addr":"10.10.11.54","etype":"IPv4","proto":"TCP","src_port":16443,"dst_port":57444,"in_if":0,"out_if":5031,"src_mac":"3c:ec:ef:fc:f7:38","dst_mac":"00:16:3e:c9:e3:eb","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":26842,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065940700426180,"sequence_num":47151,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065940700426180,"time_flow_end_ns":1709065940700426180,"bytes":1518,"packets":1,"src_addr":"10.10.11.38","dst_addr":"10.10.11.54","etype":"IPv4","proto":"TCP","src_port":16443,"dst_port":57444,"in_if":0,"out_if":5031,"src_mac":"3c:ec:ef:fc:f7:38","dst_mac":"00:16:3e:c9:e3:eb","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":27109,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065945720360778,"sequence_num":47153,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065945720360778,"time_flow_end_ns":1709065945720360778,"bytes":1518,"packets":1,"src_addr":"10.10.11.38","dst_addr":"10.10.11.54","etype":"IPv4","proto":"TCP","src_port":16443,"dst_port":57444,"in_if":0,"out_if":5031,"src_mac":"3c:ec:ef:fc:f7:38","dst_mac":"00:16:3e:c9:e3:eb","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":27412,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065950740250769,"sequence_num":47155,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065950740250769,"time_flow_end_ns":1709065950740250769,"bytes":142,"packets":1,"src_addr":"10.10.11.54","dst_addr":"10.10.11.38","etype":"IPv4","proto":"TCP","src_port":17070,"dst_port":37671,"in_if":5031,"out_if":0,"src_mac":"00:16:3e:c9:e3:eb","dst_mac":"3c:ec:ef:fc:f7:38","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":24,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":38057,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065951849809728,"sequence_num":47157,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065951849809728,"time_flow_end_ns":1709065951849809728,"bytes":1518,"packets":1,"src_addr":"10.10.11.38","dst_addr":"10.10.11.54","etype":"IPv4","proto":"TCP","src_port":16443,"dst_port":57444,"in_if":0,"out_if":5031,"src_mac":"3c:ec:ef:fc:f7:38","dst_mac":"00:16:3e:c9:e3:eb","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":28202,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065955000573971,"sequence_num":47158,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065955000573971,"time_flow_end_ns":1709065955000573971,"bytes":1518,"packets":1,"src_addr":"10.10.11.38","dst_addr":"10.10.11.54","etype":"IPv4","proto":"TCP","src_port":16443,"dst_port":57444,"in_if":0,"out_if":5031,"src_mac":"3c:ec:ef:fc:f7:38","dst_mac":"00:16:3e:c9:e3:eb","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":28256,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065955760244272,"sequence_num":47159,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065955760244272,"time_flow_end_ns":1709065955760244272,"bytes":70,"packets":1,"src_addr":"10.10.11.54","dst_addr":"10.10.11.38","etype":"IPv4","proto":"TCP","src_port":57444,"dst_port":16443,"in_if":5031,"out_if":0,"src_mac":"00:16:3e:c9:e3:eb","dst_mac":"3c:ec:ef:fc:f7:38","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":35575,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065960780292840,"sequence_num":47161,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065960780292840,"time_flow_end_ns":1709065960780292840,"bytes":70,"packets":1,"src_addr":"10.10.11.54","dst_addr":"10.10.11.38","etype":"IPv4","proto":"TCP","src_port":57444,"dst_port":16443,"in_if":5031,"out_if":0,"src_mac":"00:16:3e:c9:e3:eb","dst_mac":"3c:ec:ef:fc:f7:38","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":35727,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
{"type":"SFLOW_5","time_received_ns":1709065965800218079,"sequence_num":47163,"sampling_rate":1024,"sampler_address":"10.10.26.5","time_flow_start_ns":1709065965800218079,"time_flow_end_ns":1709065965800218079,"bytes":70,"packets":1,"src_addr":"10.10.11.38","dst_addr":"10.10.11.54","etype":"IPv4","proto":"TCP","src_port":16443,"dst_port":57444,"in_if":0,"out_if":5031,"src_mac":"3c:ec:ef:fc:f7:38","dst_mac":"00:16:3e:c9:e3:eb","src_vlan":0,"dst_vlan":0,"vlan_id":0,"ip_tos":0,"forwarding_status":0,"ip_ttl":64,"ip_flags":2,"tcp_flags":16,"icmp_type":0,"icmp_code":0,"ipv6_flow_label":0,"fragment_id":29356,"fragment_offset":0,"src_as":0,"dst_as":0,"next_hop":"","next_hop_as":0,"src_net":"0.0.0.0/0","dst_net":"0.0.0.0/0","bgp_next_hop":"","bgp_communities":[],"as_path":[],"mpls_ttl":[],"mpls_label":[],"mpls_ip":[],"observation_domain_id":0,"observation_point_id":0}
ACodingfreak commented 7 months ago

Looks like counter sampling is not supported by goflow2 but only flow sampling.

lspgn commented 6 months ago

Hello @ACodingfreak, Thank you for raising this issue. This is not a bug, the default "producer" which converts the raw flow data into something usable does not support counter. Currently there are no plans to convert counters into the protobuf.

You can try the -produce=raw mode which gives a JSON version of the sFlow:

$ go run cmd/goflow2/main.go -produce=raw -format=json -logfmt=json | jq -r '.["message"]["sequence-number"]'
{"level":"info","msg":"starting GoFlow2","time":"2024-03-23T19:14:05-07:00"}
{"blocking":false,"count":1,"hostname":"","level":"info","msg":"starting collection","port":6343,"queue_size":1000000,"scheme":"sflow","time":"2024-03-23T19:14:05-07:00","workers":2}
{"blocking":false,"count":1,"hostname":"","level":"info","msg":"starting collection","port":2055,"queue_size":1000000,"scheme":"netflow","time":"2024-03-23T19:14:05-07:00","workers":2}
47141
47142
47143
47144
47145
47146
47147
47148
47149
47150
47151
47152
47153
47154
47155
47156
47157
47158
47159
47160
47161
47162
47163