sfTypeFormat is invalid type

[huaconghub]

my sflow data decoded fail, I found when execute function getSampleInfo, sflow/decoder.go:194, the sfType is 3, not const DataFlowSample and DataCounterSample , it seem sfType always 3, and I dont know why, I checked my device sflow version is 5, and raw header is L2/L3/L4, do you know what might go wrong? thx

a sample sflow data:

0 0 0 5 0 0 0 1 10 101 4 2 0 0 0 16 0 99 162 94 144 32 242 200 0 0 0 6 0 0 0 3 0 0 0 208 2 58 30 90 0 0 0 0 0 0 0 168 0 0 19 136 253 86 63 51 0 0 0 0 0 0 0 0 0 0 0 180 0 0 0 0 0 0 0 168 0 0 0 2 0 0 0 1 0 0 0 132 0 0 0 1 0 0 0 118 0 0 0 4 0 0 0 114 4 64 169 182 177 117 108 146 191 13 38 181 129 0 1 244 8 0 69 0 0 96 34 86 64 0 64 17 183 234 111 48 61 101 117 136 62 47 5 173 230 3 0 76 55 214 66 4 64 185 172 10 2 0 4 230 86 51 181 120 167 215 47 203 169 216 1 2 138 188 1 0 1 28 64 18 252 167 63 163 242 200 91 135 0 209 29 224 242 27 127 201 23 61 191 65 16 252 167 180 139 150 109 145 133 137 102 135 198 200 180 156 223 191 0 0 0 0 3 233 0 0 0 16 0 0 1 244 0 0 0 0 0 0 1 244 0 0 0 0 0 0 0 3 0 0 0 172 2 58 30 94 0 0 0 0 0 0 0 168 0 0 19 136 253 86 63 51 0 0 0 0 0 0 0 0 0 0 0 168 0 0 0 0 0 0 0 180 0 0 0 2 0 0 0 1 0 0 0 96 0 0 0 1 0 0 0 84 0 0 0 4 0 0 0 80 4 64 169 182 177 128 12 65 233 138 36 145 8 0 69 4 0 66 171 115 64 0 49 17 5 233 124 236 110 197 111 48 61 105 3 202 3 184 0 46 13 206 97 2 64 187 250 54 162 0 246 212 0 0 183 237 197 21 220 11 189 214 28 38 48 215 1 2 216 232 161 0 1 32 64 0 252 65 0 252 0 0 3 233 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 220 2 58 30 95 0 0 0 0 0 0 0 168 0 0 19 136 253 86 63 51 0 0 0 0 0 0 0 0 0 0 0 180 0 0 0 0 0 0 0 168 0 0 0 2 0 0 0 1 0 0 0 144 0 0 0 1 0 0 0 180 0 0 0 4 0 0 0 128 4 64 169 182 177 117 108 146 191 13 38 181 129 0 1 244 8 0 69 0 0 158 24 127 64 0 64 17 84 44 111 48 61 101 223 104 65 166 3 85 199 84 0 138 193 231 38 8 32 185 188 207 93 0 15 230 181 136 154 2 0 0 171 67 227 214 1 7 222 231 46 0 3 33 1 211 220 230 214 0 94 252 252 48 36 223 213 249 132 150 38 64 117 13 110 157 209 208 166 178 218 1 77 73 191 185 9 21 72 206 200 140 79 215 191 114 21 7 73 98 66 102 136 237 168 124 51 159 0 0 3 233 0 0 0 16 0 0 1 244 0 0 0 0 0 0 1 244 0 0 0 0 0 0 0 3 0 0 0 220 2 58 30 99 0 0 0 0 0 0 0 168 0 0 19 136 253 86 63 51 0 0 0 0 0 0 0 0 0 0 0 168 0 0 0 0 0 0 0 180 0 0 0 2 0 0 0 1 0 0 0 144 0 0 0 1 0 0 0 163 0 0 0 4 0 0 0 128 4 64 169 182 177 128 12 65 233 138 36 145 8 0 69 4 0 145 233 58 64 0 50 17 235 96 61 155 137 134 111 48 61 107 3 187 3 188 0 125 174 27 81 7 64 187 52 127 7 0 196 2 29 0 182 246 66 1 105 79 97 216 131 38 185 216 1 2 125 72 5 0 1 32 32 38 252 152 102 63 194 230 88 54 175 153 97 13 132 4 76 253 40 30 54 196 215 247 150 238 131 81 194 220 70 238 185 35 132 212 122 69 189 133 159 33 41 252 152 65 90 209 100 60 75 64 239 213 0 0 3 233 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 220 2 58 30 100 0 0 0 0 0 0 0 168 0 0 19 136 253 86 63 51 0 0 0 0 0 0 0 0 0 0 0 180 0 0 0 0 0 0 0 168 0 0 0 2 0 0 0 1 0 0 0 144 0 0 0 1 0 0 0 133 0 0 0 4 0 0 0 128 4 64 169 182 177 117 108 146 191 13 38 181 129 0 1 244 8 0 69 0 0 111 137 240 64 0 64 17 111 196 111 48 61 101 111 50 37 2 3 88 107 230 0 91 210 104 50 5 64 185 134 166 0 0 116 97 255 15 16 216 140 215 121 80 168 214 1 2 100 88 0 0 1 28 64 27 252 170 222 213 109 50 71 52 7 30 44 91 86 56 140 248 159 23 86 241 32 81 129 181 94 99 117 255 65 22 252 170 91 88 82 83 4 9 105 227 75 203 204 58 194 239 104 125 184 64 130 118 0 0 3 233 0 0 0 16 0 0 1 244 0 0 0 0 0 0 1 244 0 0 0 0 0 0 0 3 0 0 0 196 2 58 30 101 0 0 0 0 0 0 0 168 0 0 19 136 253 86 63 51 0 0 0 0 0 0 0 0 0 0 0 168 0 0 0 0 0 0 0 180 0 0 0 2 0 0 0 1 0 0 0 120 0 0 0 1 0 0 0 106 0 0 0 4 0 0 0 102 4 64 169 182 177 128 12 65 233 138 36 145 8 0 69 4 0 88 225 234 64 0 55 17 68 2 39 159 73 106 111 48 61 107 194 255 5 176 0 68 148 69 193 3 64 169 34 86 0 0 213 139 88 5 211 13 202 216 1 2 0 8 0 0 1 24 64 18 252 168 120 88 246 77 109 76 234 159 196 107 29 151 55 179 103 91 191 65 12 252 168 28 84 108 99 177 13 7 163 153 49 183 1 0 0 0 3 233 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[mehrdadrad]

@huaconghub what's vFlow version?

vflow -version

[huaconghub]

@mehrdadrad vflow 0.6.5 I git clone from master branch

@huaconghub what's vFlow version?

vflow -version

[mehrdadrad]

interesting, can you let me know the Go version, OS and arch?

[huaconghub]

interesting, can you let me know the Go version, OS and arch?

Go: go1.11.2 linux/amd64 OS: Ubuntu 16.04.4 LTS xenial

[mehrdadrad]

• add more logging after line 188 at vflow/sflow.go

  logger.Println(err)

• run the vflow in the vebose mode under sflow load and send here the log file and console logs

[huaconghub]

• add more logging after line 188 at vflow/sflow.go

logger.Println(err)

• run the vflow in the vebose mode under sflow load and send here the log file and console logs

I add a log on verbose mode, error is nil:

[vflow] 2018/12/08 14:08:40 sflow.go:190: 0%!(EXTRA <nil>)

[huaconghub]

I found this tool can precisely decode my sflow data, https://github.com/sflow/sflowtool

[mehrdadrad]

the vflow parses v5 as the error is nil but can not see any samples! (raw or counters) can you paste the sflowtool output here? what's your device?

[huaconghub]

the vflow parses v5 as the error is nil but can not see any samples! (raw or counters) can you paste the sflowtool output here? what's your device?

Sample data decode by sflowtool (some field replace with *):

{"datagramSourceIP":"*","datagramSize":"1296","unixSecondsUTC":"1544407110","localtime":"2018-12-10T09:58:30+0800","datagramVersion":"5","agentSubId":"16","agent":"10.101.4.2","packetSequenceNo":"11048351","sysUpTime":"2638308408","samplesInPacket":"6","samples":[{"sampleType_tag":"0:3","sampleType":"FLOWSAMPLE","sampleSequenceNo":"63024708","sourceId":"0:168","meanSkipCount":"5000","samplePool":"3761648742","dropEvents":"0","inputPort":"180","outputPort":"168","elements":[{"flowBlock_tag":"0:1","flowSampleType":"HEADER","headerProtocol":"1","sampledPacketSize":"218","strippedBytes":"4","headerLen":"128","headerBytes":"*","dstMAC":"*","srcMAC":"*","decodedVLAN":"500","decodedPriority":"0","IPSize":"196","ip.tot_len":"196","srcIP":"*","dstIP":"*","IPProtocol":"17","IPTOS":"0","IPTTL":"64","IPID":"45749","UDPSrcPort":"954","UDPDstPort":"960","UDPBytes":"176"},{"flowBlock_tag":"0:1001","extendedType":"SWITCH","in_vlan":"500","in_priority":"0","out_vlan":"500","out_priority":"0"}]},{"sampleType_tag":"0:3","sampleType":"FLOWSAMPLE","sampleSequenceNo":"63024710","sourceId":"0:168","meanSkipCount":"5000","samplePool":"3761648742","dropEvents":"0","inputPort":"180","outputPort":"168","elements":[{"flowBlock_tag":"0:1","flowSampleType":"HEADER","headerProtocol":"1","sampledPacketSize":"1522","strippedBytes":"4","headerLen":"128","headerBytes":"*","dstMAC":"*","srcMAC":"*","decodedVLAN":"500","decodedPriority":"0","IPSize":"1500","ip.tot_len":"1500","srcIP":"*","dstIP":"*","IPProtocol":"6","IPTOS":"0","IPTTL":"64","IPID":"1533","TCPSrcPort":"9191","TCPDstPort":"45132","TCPFlags":"24"},{"flowBlock_tag":"0:1001","extendedType":"SWITCH","in_vlan":"500","in_priority":"0","out_vlan":"500","out_priority":"0"}]},{"sampleType_tag":"0:3","sampleType":"FLOWSAMPLE","sampleSequenceNo":"63024712","sourceId":"0:168","meanSkipCount":"5000","samplePool":"3761648742","dropEvents":"0","inputPort":"168","outputPort":"180","elements":[{"flowBlock_tag":"0:1","flowSampleType":"HEADER","headerProtocol":"1","sampledPacketSize":"76","strippedBytes":"4","headerLen":"72","headerBytes":"*","dstMAC":"*","srcMAC":"*","IPSize":"58","ip.tot_len":"58","srcIP":"*","dstIP":"*","IPProtocol":"17","IPTOS":"116","IPTTL":"54","IPID":"9997","UDPSrcPort":"6931","UDPDstPort":"1456","UDPBytes":"38"},{"flowBlock_tag":"0:1001","extendedType":"SWITCH","in_vlan":"0","in_priority":"0","out_vlan":"0","out_priority":"0"}]},{"sampleType_tag":"0:3","sampleType":"FLOWSAMPLE","sampleSequenceNo":"63024714","sourceId":"0:168","meanSkipCount":"5000","samplePool":"3761648742","dropEvents":"0","inputPort":"180","outputPort":"168","elements":[{"flowBlock_tag":"0:1","flowSampleType":"HEADER","headerProtocol":"1","sampledPacketSize":"160","strippedBytes":"4","headerLen":"128","headerBytes":"*","dstMAC":"*","srcMAC":"*","decodedVLAN":"500","decodedPriority":"0","IPSize":"138","ip.tot_len":"138","srcIP":"*","dstIP":"*","IPProtocol":"17","IPTOS":"0","IPTTL":"64","IPID":"4689","UDPSrcPort":"858","UDPDstPort":"55590","UDPBytes":"118"},{"flowBlock_tag":"0:1001","extendedType":"SWITCH","in_vlan":"500","in_priority":"0","out_vlan":"500","out_priority":"0"}]},{"sampleType_tag":"0:3","sampleType":"FLOWSAMPLE","sampleSequenceNo":"63024716","sourceId":"0:168","meanSkipCount":"5000","samplePool":"3761648742","dropEvents":"0","inputPort":"180","outputPort":"168","elements":[{"flowBlock_tag":"0:1","flowSampleType":"HEADER","headerProtocol":"1","sampledPacketSize":"85","strippedBytes":"4","headerLen":"81","headerBytes":"*","dstMAC":"*","srcMAC":"*","decodedVLAN":"500","decodedPriority":"0","IPSize":"63","ip.tot_len":"63","srcIP":"*","dstIP":"*","IPProtocol":"17","IPTOS":"0","IPTTL":"64","IPID":"47957","UDPSrcPort":"952","UDPDstPort":"975","UDPBytes":"43"},{"flowBlock_tag":"0:1001","extendedType":"SWITCH","in_vlan":"500","in_priority":"0","out_vlan":"500","out_priority":"0"}]},{"sampleType_tag":"0:3","sampleType":"FLOWSAMPLE","sampleSequenceNo":"63024718","sourceId":"0:168","meanSkipCount":"5000","samplePool":"3761648742","dropEvents":"0","inputPort":"180","outputPort":"168","elements":[{"flowBlock_tag":"0:1","flowSampleType":"HEADER","headerProtocol":"1","sampledPacketSize":"1322","strippedBytes":"4","headerLen":"128","headerBytes":"*","dstMAC":"*","srcMAC":"*","decodedVLAN":"500","decodedPriority":"0","IPSize":"1300","ip.tot_len":"1300","srcIP":"*","dstIP":"*","IPProtocol":"6","IPTOS":"0","IPTTL":"64","IPID":"23927","TCPSrcPort":"600","TCPDstPort":"30536","TCPFlags":"16"},{"flowBlock_tag":"0:1001","extendedType":"SWITCH","in_vlan":"500","in_priority":"0","out_vlan":"500","out_priority":"0"}]}]}

[mehrdadrad]

everything looks good, I think it decodes and you need to consume them from vFlow. what's your messaging queue? can you send me the below log in verbose mode and messaging queue configuration? /var/log/vflow.log /etc/vflow/mq.conf

[huaconghub]

everything looks good, I think it decodes and you need to consume them from vFlow. what's your messaging queue? can you send me the below log in verbose mode and messaging queue configuration? /var/log/vflow.log /etc/vflow/mq.conf

I check the log many times, unfortunately, no helpful info, just:

rcvd sflow data from: xx.xx.xx.xx:**, size: 1396 bytes

mq is kafka, config file only kafka brokers, and netflow send msg succuess, so mq is fine;

[mehrdadrad]

once you ran the vflow for a while, pls send me the below: curl localhost:8081/flow | json_xs

[huaconghub]

json_xs

{
  "StartTime": 1544585675,
  "IPFIX": {
    "UDPQueue": 0,
    "UDPMirrorQueue": 0,
    "MessageQueue": 0,
    "UDPCount": 0,
    "DecodedCount": 0,
    "MQErrorCount": 0,
    "Workers": 0
  },
  "SFlow": {
    "UDPQueue": 0,
    "MessageQueue": 0,
    "UDPCount": 284112,
    "DecodedCount": 0,
    "MQErrorCount": 0,
    "Workers": 200
  },
  "NetflowV9": {
    "UDPQueue": 0,
    "MessageQueue": 0,
    "UDPCount": 0,
    "DecodedCount": 0,
    "MQErrorCount": 0,
    "Workers": 0
  }
}

[huaconghub]

device: H3C-S6800-54QF