Open raghurampai opened 5 years ago
@raghurampai it seems very generic: while the library can be improved to support it while this may not be used when producing the protobuf abstracted flows. So far no plan of supporting them.
What's the use case? What are the fields produced? What are the devices producing it? Do you have a pcap?
Hi just jumping in We are using dedicated Netflow Probes like Flowmon and others. These are capable of extracting additional information such as DNS requests, HTTP information etc. This is very useful in open high performance environments where firewall, proxy etc. are no option. This data is then transmitted in IPFIX like any other network traffic information. Using the corresponding Enterprise ID (IANA is 0) and Field IDs configurations, this allows IPFIX to be used as a widely supported, very flexible data transmission container. Ideally the enterprise field definitions would not be hard coded, but configurable. The CESNET IPFIXcol implements such features.
Cheers.
@mathiaske Got it, thank you for the details. I believe this is beyond the scope of GoFlow which is meant to produce a unique flow format for network samples. The decoding library could definitely benefit from this though. Could you send some samples? Actually, sFlow also supports things like DNS and HTTP, in which case, it may be interesting to have in GoFlow.
@lspgn, sorry for the delay in response. I don't have the pcap currently. The use case at high level is similar to what @mathiaske mentioned. To describe further, the monitoring of BNG (broadband subscriber services) will have some enterprise specific information elements, which basically needs to be processed- https://www.juniper.net/documentation/en_US/junos/topics/concept/ipfix-mediation-bng.html
Thanks!
@raghurampai thank you for the details! I will definitely look into it. Do you have a spec detailing the enterprise specifics fields?
We currently use some Flowmon Probes -> CESNET, Flowmon (former Inveatec) fields We also intend to have our own fields in the future for some use cases -> it would be greate if this would be configurable (but please not in xml) Example: https://github.com/CESNET/ipfixcol/blob/master/base/config/ipfix-elements.xml
Below link has the enterprise specific fields- https://www.juniper.net/documentation/en_US/junos/topics/reference/general/logging-dictionary-template-types-tdf.html
Yes, it would be great to have option of some sort of configurable file for goflow decoder to be able to understand the enterprise specific fields. The decoder could then decode these additional fields.
Raising this ticket to request support for features new to IPFIX as compared to Netflow V9- 1) Support for variable length information element- ====== snip from RFC 7011 ========
Variable-Length Information Element
The IPFIX Template mechanism is optimized for fixed-length Information Elements [RFC7012]. Where an Information Element has a variable length, the following mechanism MUST be used to carry the length information for both the IANA-assigned and enterprise-specific Information Elements.
In the Template Set, the Information Element Field Length is recorded as 65535. This reserved length value notifies the Collecting Process that the length value of the Information Element will be carried in the Information Element content itself.
In most cases, the length of the Information Element will be less than 255 octets. The following length-encoding mechanism optimizes the overhead of carrying the Information Element length in this more common case. The length is carried in the octet before the Information Element, as shown in Figure R.
The length may also be encoded into 3 octets before the Information Element, allowing the length of the Information Element to be greater than or equal to 255 octets. In this case, the first octet of the Length field MUST be 255, and the length is carried in the second and third octets, as shown in Figure S.
====================================
2) Support for enterprise specific information elements- ==== snip from RFC 7011 ===== 3.2. Field Specifier Format
Vendors need the ability to define proprietary Information Elements, because, for example, they are delivering a pre-standards product, or the Information Element is in some way commercially sensitive. This section describes the Field Specifier format for both IANA-registered Information Elements [IANA-IPFIX] and enterprise-specific Information Elements.
The Field Specifier format is shown in Figure G.
Where:
E
Information Element identifier
Field Length
Enterprise Number
========================
Saw that these features are not supported currently. Please let me know if any plans to support these.
Thanks!