Cannot parse the NetStream protocol normally

[ChaoHsupin]

Hi！I'm having trouble parsing Huawei netstream，Different from the packet loss problem I encountered #216

goflow2 version：v2.0.0 I config netflow channel in goflow2 , to receive netstream udp packets，But none of the packets can be parsed. There are no related exception logs.

this is my tcpdum pcap file: https://github.com/ChaoHsupin/open-file/blob/main/netstream_tcpdump.pcap

[lspgn]

Hi @ChaoHsupin, Thank you for the packet capture. It seems to be the v5 version of NetFlow. Could you show me what's the command you're running for GoFlow2?

I was able to decode the packets on v2.0.0 and latest main.

[ChaoHsupin]

Hi @ChaoHsupin, Thank you for the packet capture. It seems to be the v5 version of NetFlow. Could you show me what's the command you're running for GoFlow2?

I was able to decode the packets on v2.0.0 and latest main. @lspgn Thanks for your help!

This is my cmd:

./goflow2_2_0 -listen=sflow://:6343?count=3,netflow://:2055?count=3 -transport.kafka.brokers=10.12.6.7:9094,10.12.6.8:9094,10.12.6.9:9094,10.12.6.10:9094,10.12.6.11:9094,10.12.6.12:9094,10.12.6.13:9094  -transport.kafka.version=0.11.0.2 -transport=kafka -transport.kafka.topic=flows -format=json

[lspgn]

Does it log anything if you run it with:

./goflow2_2_0 -listen="sflow://:6343?count=3,netflow://:2055?count=3" -format=json -transport.file= -transport=file

Can you provide me with the Prometheus metrics on http://localhost:8080/metrics ?

[ChaoHsupin]

@lspgn Thanks, This is metrics:https://github.com/ChaoHsupin/open-file/blob/main/goflow2_metrics.log

[lspgn]

Thank you for the metrics. I'm seeing it's decoding the netstream under netflow (v5) with around 800 packets. Do you still see the issue?

[ChaoHsupin]

@lspgn Thanks for your advice! I consumed kafka data in flink. in my debug ,I found that time_received_ns be parsed with 0. this 0 caused my program to not work properly.

this is a msg was parsed from kafka by debug :

{
    "type": "NETFLOW_V5",
    "time_received_ns": 0,
    "sequence_num": 12105295,
    "sampling_rate": 0,
    "sampler_address": "10.17.101.2",
    "time_flow_start_ns": 1697079319761900544,
    "time_flow_end_ns": 1697094085761900544,
    "bytes": 1554,
    "packets": 6,
    "src_addr": "10.101.46.8",
    "dst_addr": "180.184.142.59",
    "etype": "IPv4",
    "proto": "TCP",
    "src_port": 8443,
    "dst_port": 59086,
    "in_if": 37,
    "out_if": 0,
    "src_mac": "00:00:00:00:00:00",
    "dst_mac": "00:00:00:00:00:00",
    "src_vlan": 0,
    "dst_vlan": 0,
    "vlan_id": 0,
    "ip_tos": 0,
    "forwarding_status": 0,
    "ip_ttl": 0,
    "tcp_flags": 24,
    "icmp_type": 0,
    "icmp_code": 0,
    "ipv6_flow_label": 0,
    "fragment_id": 0,
    "fragment_offset": 0,
    "src_as": 0,
    "dst_as": 0,
    "next_hop": "116.172.66.65",
    "next_hop_as": 0,
    "src_net": "10.101.44.0/22",
    "dst_net": "0.0.0.0/0",
    "bgp_next_hop": "",
    "bgp_communities":
    [],
    "as_path":
    [],
    "mpls_ttl":
    [],
    "mpls_label":
    [],
    "mpls_ip":
    [],
    "observation_domain_id": 0,
    "observation_point_id": 0
}

[lspgn]

Missing time_received_ns has been fixed in https://github.com/netsampler/goflow2/commit/026fef546e9f116630e7abc658c859295f83cc19 (#209). I probably should do a release soon.

[ChaoHsupin]

@lspgn Ha ha! Too coincidental, thank you！