Incorrect calculation of spanning tree packet

[pwp333]

We found that netflow can report wrong flow if spanning tree protocol is enabled. It looks that 1 - Spanning Tree packets are treated as IPv6 packets(HOPOPT/0) in emitted flows 2 - Sometimes flows for such packets have abnormally large IN_BYTES/OUT_BYTES 3 - Flows with truncated Application names, Wrong IP versions (0,7 or 111), Wrong direction (21, 12). 4 - Some flows end up with a negative time Some samples are attached.

Could you please help to check code? Thanks.

[aabc]

Do you have example packets?

[jcherkas]

one thing we did find is that the netflow/ipfix template is sending:
[
"uint32",
"tcpOptions"
]
but the rfc defines tcpOptions as uint64 our netflow collector also defines tcpOptions as 209:

• :uint64
• :tcpOptions This could explain incorrect values showing up.

[aabc]

So this is resolved?

[aabc]

Length of the data field is specified in template. See http://tools.ietf.org/html/rfc5101

6.2.  Reduced Size Encoding of Integer and Float Types

   Information Elements containing integer, string, float, and
   octetarray types in the information model MAY be encoded using fewer
   octets than those implied by their type in the information model
   definition [RFC5102], [...]

   For instance, the information model [RFC5102] defines byteCount as an
   unsigned64 type, which would require 64 bits.  However, if the
   Exporter will never locally encounter the need to send a value larger
   than 4294967295, it may chose to send the value instead as an
   unsigned32. [...]

   This behavior is indicated by the Exporter by specifying a type size
   with a smaller length than that associated with the assigned type of
   the Information Element.  In the example above, the Exporter would
   place a length of 4 versus 8 in the Template.

     [...]  The reduction in size can be to
   any number of octets smaller than the original type if the data value
   still fits, i.e., so that only leading zeroes are dropped.  For
   example, an unsigned64 can be reduced in size to 7, 6, 5, 4, 3, 2, or
   1 octet(s).

[jcherkas]

Thanks for the reply. The issue we are seeing has been reported before https://github.com/logstash-plugins/logstash-codec-netflow/issues/123 and was addressed in 2018.

This is issue can be seen if using different netflow agents/exporters but we use ipt_netflow exclusively and still seeing our collector report: Template length exceeds flowset length, skipping {:template_id=>259, :template_length=>59, :record_length=>58}

[jcherkas]

This fixes the issue completely engine_id=number

• Observation Domain ID (on IPFIX, Source Id on NetFlow v9, or Engine Id on NetFlow v5) value to be exported. This may help your collector to distinguish between multiple exporters. On Netflow v9 and IPFIX this value is 32-bit on NetFlow v5 only 8 low bits are significant. Default value is 0.

Giving unique engine_id to each ipt_netflow exporter prevents template collisions at the collector and avoids flow parsing issues.

[aabc]

Thanks for investigating it!