search

phaag / nfdump

Netflow processing tools
Other
771 stars 202 forks source link

Multiple elements of the same type are not supported in one template #320

Closed nward closed 1 year ago

nward commented 2 years ago

After spending some time digging through the code, I fear resolving this may be a bit of an architectural change, but, better to get it on the board for future I figure.

RFC7011 (and RFC5101) require that collectors support multiple information elements of the same type in a template. This is uncommon, which is why I suspect it is not supported here.

This is something we see in JunOS CGNAT records, documented here: https://www.juniper.net/documentation/us/en/software/junos/flow-monitoring/topics/concept/services-logging-flowmonitoring-format-nat-events.html#understanding-nat-event-logging-in-flow-monitoring-format-on-an-mx-series-router-or-nfx__flow_template_NAT44_port_allocation

Note the multiple occurrences of observationTimeMilliseconds in tables 11 and 12, with different meanings. In both of these, the first occurrence is the event time, the second occurrence is the allocation time.

nfcapd appears uses the latter occurrence - which means for deallocation events, we see the allocation time, rather than the event time (the first occurrence).

phaag commented 2 years ago

Ahh - what a wired definition! Indeed, this is not handled by nfdump. The quick and dirty solution would be to take the first occurrence and ignore the second. This does not solve the problem and still ignores the double occurrence. But a clean handling requires more time to fix, as the internal architecture does not handle this.

nward commented 2 years ago

Yeah it's weird alright. I've not seen other equipment do this, but I guess it's legal.

We can work around it by matching up allocation events, but it's not perfect.

Anyway as I say, getting it on the board for a longer term fix :-)

phaag commented 1 year ago

I will close the ticket for now.. Not sure, when this gets implemented.