Support for named element ID's in Netflow and IPfix

[fatwookie]

I've got a feature request for vflow. It would be very convenient if the Netflow and IPfix could be configured to output decoded named IANA elements. Sort of like the sFlow module currently does. This way the JSON output could be ingested into a logstash shipper (through an MQ).

[niltooth]

I second this.

[Slepwin]

It would be great.

[Slepwin]

Looks like ipfix.elements on /etc/vflow/ doesn't work anymore from: https://github.com/VerizonDigital/vflow/issues/68

[mehrdadrad]

it means the file is not exist through rpm or it doesn't load? https://github.com/VerizonDigital/vflow/blob/master/scripts/ipfix.elements

[Slepwin]

it means the file is not exist through rpm or it doesn't load? https://github.com/VerizonDigital/vflow/blob/master/scripts/ipfix.elements

doesn't load, currently i have installed the latest version of RPM and copy https://github.com/VerizonDigital/vflow/blob/master/scripts/ipfix.elements to /etc/vflow/ with same rights as mq and vflow.conf and restart vflow service, i see in kafka topics only element ID not names. Also doesn't find elements file for Netflow v9.

[Slepwin]

it means the file is not exist through rpm or it doesn't load? https://github.com/VerizonDigital/vflow/blob/master/scripts/ipfix.elements

@mehrdadrad could you please describe how i can configure vflow with ipfix.elements if it works on current release.

[mehrdadrad]

@Slepwin Once you copied the file to /etc/vflow or your specified config directory, you can add enterprise elements:

enterprise-number:
  element-id:
  - name
  - datatype

you can find those information from the vendor documentation. if you want to add to the zero, you can append the new elements which defined at IANA at the end of the current ipfix.elements (after element number 433) the IANA https://www.iana.org/assignments/ipfix/ipfix.xhtml

 433:
  - ignoredLayer2FrameTotalCount
  - unsigned64

[Slepwin]

@Slepwin Once you copied the file to /etc/vflow or your specified config directory, you can add enterprise elements:

enterprise-number:
  element-id:
  - name
  - datatype

you can find those information from the vendor documentation. if you want to add to the zero, you can append the new elements which defined at IANA at the end of the current ipfix.elements (after element number 433) the IANA https://www.iana.org/assignments/ipfix/ipfix.xhtml

 433:
  - ignoredLayer2FrameTotalCount
  - unsigned64

Hi @mehrdadrad my question not about enterprise elements, i see in kafka topics only element ID not names: {"AgentID":"10.0.13.1","Header":{"Version":10,"Length":105,"ExportTime":1610638197,"SequenceNo":1618410321,"DomainID":524288},"DataSets":[[{"I":8,"V":"10.15.223.228"},{"I":12,"V":"10.62.185.40"},{"I":5,"V":0},{"I":4,"V":6},{"I":7,"V":35634},{"I":11,"V":10050},{"I":32,"V":0},{"I":10,"V":631},{"I":58,"V":2011},{"I":9,"V":16},{"I":13,"V":16},{"I":16,"V":65001},{"I":17,"V":65002},{"I":15,"V":"10.13.254.5"},{"I":6,"V":"0x10"},{"I":14,"V":584},{"I":1,"V":40},{"I":2,"V":1},{"I":52,"V":60},{"I":53,"V":60},{"I":152,"V":1610638137600},{"I":153,"V":1610638137600},{"I":136,"V":2},{"I":61,"V":255},{"I":243,"V":0},{"I":245,"V":0},{"I":54,”V”:0}]]}

How i can achieve this format on vflow side: {"AgentID"=>"10.0.13.1", "Header"=>{"Version"=>10, "Length"=>105, "ExportTime"=>1610638197, "SequenceNo"=>1618410321, "DomainID"=>524288}, "DataSets"=>[{"sourceIPv4Address"=>"10.15.223.228"}, {"destinationIPv4Address"=>"10.62.185.40"}, {"ipClassOfService"=>0}, {"protocolIdentifier"=>6}, {"sourceTransportPort"=>35634}, {"destinationTransportPort"=>10050}, {"icmpTypeCodeIPv4"=>0}, {"ingressInterface"=>631}, {"vlanId"=>2011}, {"sourceIPv4PrefixLength"=>16}, {"destinationIPv4PrefixLength"=>16}, {"bgpSourceAsNumber"=>65001}, {"bgpDestinationAsNumber"=>65002}, {"ipNextHopIPv4Address"=>"10.13.254.5"}, {"tcpControlBits"=>"0x10"}, {"egressInterface"=>584}, {"octetDeltaCount"=>40}, {"packetDeltaCount"=>1}, {"minimumTTL"=>60}, {"maximumTTL"=>60}, {"flowStartMilliseconds"=>1610638137600}, {"flowEndMilliseconds"=>1610638137600}, {"flowEndReason"=>2}, {"flowDirection"=>255}, {"dot1qVlanId"=>0}, {"dot1qCustomerVlanId"=>0}, {"fragmentIdentification"=>0}]}

[mehrdadrad]

@Slepwin It doesn't!