sFlow messages to Kafka MQ broken

[Slepwin]

Hi,

I have 0.9.0 version in k8s setup with ipfix/netflow v9 and sflow workers. IPFIX and Netflow v9 work fine and i see messages in related kafka topics, but don't see vflow.sflow topics in kafka. With only sflow enabled workers i see packets in tcpdump with length 0 on kafka nodes. Moreover with verbose: true in vflow.conf i can see successfully parsed sflow data like:

kubectl logs -f pod/vflow-66d586c8f6-jnn56
vFlow running with PID 9.
[vflow] 2021/03/01 13:51:33 the full logging enabled
[vflow] 2021/03/01 13:51:33 options.go:217: Welcome to vFlow v.0.9.0 Apache License 2.0
[vflow] 2021/03/01 13:51:33 options.go:218: Copyright (C) 2018 Verizon. github.com/VerizonDigital/vflow
[vflow] 2021/03/01 13:51:33 ipfix.go:95: ipfix has been disabled
[vflow] 2021/03/01 13:51:33 stats.go:161: starting prometheus http server ...
[vflow] 2021/03/01 13:51:33 netflow_v5.go:88: netflowv5 has been disabled
[vflow] 2021/03/01 13:51:33 netflow_v9.go:91: netflowv9 has been disabled
[vflow] 2021/03/01 13:51:33 sflow.go:117: sFlow is running (UDP: listening on [::]:6343 workers#: 100)
[vflow] 2021/03/01 13:51:33 sarama.go:142: start producer: Kafka, brokers: [sjc06-c01-klq01:9092 sjc06-c01-klq02:9092], topic: vflow.sflow
[vflow] 2021/03/01 13:51:34 sflow.go:186: rcvd sflow data from: 10.12.100.219:2971, size: 160 bytes
[vflow] 2021/03/01 13:51:34 sflow.go:186: rcvd sflow data from: 10.12.100.219:2971, size: 148 bytes
[vflow] 2021/03/01 13:51:34 sflow.go:186: rcvd sflow data from: 10.12.100.219:2971, size: 160 bytes
[vflow] 2021/03/01 13:51:34 sflow.go:219: {"Version":5,"IPVersion":1,"AgentSubID":0,"SequenceNo":1100422010,"SysUpTime":1489308996,"SamplesNo":1,"Samples":[{"SequenceNo":303836076,"SourceID":0,"SamplingRate":100,"SamplePool":317572528,"Drops":0,"Input":59,"Output":62,"RecordsNo":1,"Records":{"RawHeader":{"L2":{"SrcMAC":"00:09:0f:09:00:1d","DstMAC":"7c:e2:ca:5a:84:72","Vlan":0,"EtherType":2048},"L3":{"Version":4,"TOS":0,"TotalLen":52,"ID":32013,"Flags":0,"FragOff":0,"TTL":59,"Protocol":6,"Checksum":34901,"Src":"10.16.21.1","Dst":"10.15.17.66"},"L4":{"SrcPort":9092,"DstPort":37032,"DataOffset":8,"Reserved":0,"Flags":16}}}}],"Counters":[],"IPAddress":"10.12.100.219","ColTime":1614606694}
[vflow] 2021/03/01 13:51:34 sflow.go:219: {"Version":5,"IPVersion":1,"AgentSubID":0,"SequenceNo":1100422011,"SysUpTime":1489308996,"SamplesNo":1,"Samples":[{"SequenceNo":127725068,"SourceID":0,"SamplingRate":100,"SamplePool":4182842808,"Drops":0,"Input":67,"Output":0,"RecordsNo":1,"Records":{"RawHeader":{"L2":{"SrcMAC":"88:e0:f3:7f:8c:c2","DstMAC":"00:09:0f:09:00:20","Vlan":0,"EtherType":2048},"L3":{"Version":4,"TOS":0,"TotalLen":52,"ID":29996,"Flags":0,"FragOff":0,"TTL":60,"Protocol":6,"Checksum":32729,"Src":"10.15.37.100","Dst":"10.16.16.60"},"L4":{"SrcPort":1777,"DstPort":34750,"DataOffset":8,"Reserved":0,"Flags":16}}}}],"Counters":[],"IPAddress":"10.12.100.219","ColTime":1614606694}
[vflow] 2021/03/01 13:51:34 sflow.go:219: {"Version":5,"IPVersion":1,"AgentSubID":0,"SequenceNo":1100422009,"SysUpTime":1489308996,"SamplesNo":1,"Samples":[{"SequenceNo":303836075,"SourceID":0,"SamplingRate":100,"SamplePool":317572428,"Drops":0,"Input":62,"Output":0,"RecordsNo":1,"Records":{"RawHeader":{"L2":{"SrcMAC":"b8:c2:53:5c:f4:40","DstMAC":"00:09:0f:09:00:1d","Vlan":0,"EtherType":2048},"L3":{"Version":4,"TOS":0,"TotalLen":40,"ID":27666,"Flags":0,"FragOff":0,"TTL":63,"Protocol":6,"Checksum":28829,"Src":"10.15.36.90","Dst":"10.24.38.160"},"L4":{"SrcPort":58241,"DstPort":1234,"DataOffset":5,"Reserved":0,"Flags":16}}}}],"Counters":[],"IPAddress":"10.12.100.219","ColTime":1614606694} 

How can i troubleshoot messages from vflow? As i say before ipfix and netflow v9 works pretty fine on the same setup.

[mehrdadrad]

@Slepwin you should see the kafka errors at logging. try w/ a single broker and see how it works!

[Slepwin]

issue on kafka side.