Problem with Juniper IPFIX

[broonu]

This is a fresh install of ELK Stack 5.6.3, and I receive this errors in logstash-plain.log:

1) Can't (yet) decode flowset id 512 from observation domain id 0, because no template to decode it with has been received. This message will usually go away after 1 minute. (This is error is persistent for hours.)

2) Invalid netflow packet received (End of file reached)

[2017-10-24T17:09:37,633][WARN ][logstash.codecs.netflow  ] Invalid netflow packet received (End of file reached)
[2017-10-24T17:09:37,633][WARN ][logstash.codecs.netflow  ] Can't (yet) decode flowset id 512 from observation domain id 0, because no template to decode it with has been received. This message will usually go away after 1 minute.

However, the elasticsearch database is feeded with some information, showed above. The only error I can see in the information received is wrong source and destionation AS numbers in some flows, not in all of them.

• Version: logstash 5.6.3

• Operating System: Debian 8

• Config File (if you have sensitive info, please remove it):

  
  input {
  udp {
  port  => 2055
  codec => netflow {
     versions => [10]
     target => ipfix
  }
  type => "netflow-pttsp"
  }
  }

output { if [type] == "netflow-pttsp" { stdout { codec => rubydebug } elasticsearch { hosts => "localhost:9200" index => "netflow-pttsp-%{+YYYY.MM.dd}" manage_template => true } } }

- Sample Data:

{ "_index": "netflow-pttsp-2017.10.24", "_type": "netflow-pttsp", "_id": "AV9PVM1mBUAoOlu2meMl", "_version": 1, "_score": null, "_source": { "@version": "1", "host": "10.0.0.5", "ipfix": { "destinationIPv4Address": "X.X.X.X", "destinationTransportPort": 62709, "icmpTypeCodeIPv4": 0, "tcpControlBits": 0, "sourceIPv4Address": "X.X.X.X", "bgpDestinationAsNumber": 4294967295, "ipClassOfService": 0, "ingressInterface": 608, "version": 10, "packetDeltaCount": 2, "flowEndReason": 2, "bgpSourceAsNumber": 4294967295, "protocolIdentifier": 17, "destinationIPv4PrefixLength": 32, "sourceIPv4PrefixLength": 24, "egressInterface": 583, "octetDeltaCount": 96, "ipNextHopIPv4Address": "X.X.X.X", "sourceTransportPort": 1101, "flowEndMilliseconds": "2017-10-24T17:01:09.368Z", "flowStartMilliseconds": "2017-10-24T16:58:33.365Z" }, "@timestamp": "2017-10-24T17:01:45.000Z", "type": "netflow-pttsp" }, "fields": { "ipfix.flowStartMilliseconds": [ 1508864313365 ], "@timestamp": [ 1508864505000 ], "ipfix.flowEndMilliseconds": [ 1508864469368 ] }, "sort": [ 1508864505000 ] }

[jorritfolmer]

Can you provide a .pcap via email (see my profile) so I can take a closer look at it?

[jorritfolmer]

Haven't seen a pcap yet. Closing. If I've missed it please include an identifier here that I can use to search my mailbox with :)

[rhasti]

I have sent captures including flow set id's 512 and 513