phaag
commented
6 years ago The time in netflow records is not an easy one. Depending on the netflow version IPFIX or v5, v9 you have different models how the time is encoded. nfdump should decode the time correctly, if time information is sent by the exporting device. Almost all time information is relative to UNIX time ( in sec, msec, usec, absolute or relative to boot time). Exporting the time is one part, displaying another one. Actually you should never see a timestamp < 1970. I suspect a time zone issue at some point. If no time is exported (the time stamp is 0) you should see 1.1.1970, optionally corrected by a time zone difference or asynchronous clocks. Feel free to send me the utm08-2.pcap to check.
I had previously posted this in the "nfsen" mailing list. Maybe that was the wrong place.
I am using nfcapd with a couple devices, among them an OpenBSD "pflow" device, which is IPFIX (Netflow 10), and a Sophos UTM 9.4-08 Software Appliance, which makes the same claim to support IPFIX.
Nfcapd supports the OpenBSD "pflow" device just fine and captures all the data as expected according to the templates, but Nfcapd does not interpret the flow record times from the Sophos UTM appliance. It displays them as "12-31-69".
Upon request, I will furnish anyone interested with some sample captures showing the main observation I see that may be relevant and that is the difference in length of the flow record timestamps:
Both captures have have 32-bit IPFIX Message Header Export Timestamps.
Please note the template packets for each capture describes the timestamps with accurate lengths (8-byte and 4-byte, respectively).
Am I reading the spec correctly? I am referring to RFC7011, section 5.1-6.1.8. It seems to leave open using either absolute or relative times, as well as dateTimeSeconds or dateTimeMilliseconds or dateTimeMicroSeconds. If the RFC allows for each all of those types of timestamps, I am wondering which are supported by nfcapd? I can't find any reference in the docs or on this list to show me which of those are supported by nfcapd. Is RFC7011 the correct reference I should be using here?
What RFCs and or other documents are recommended as a guide to debugging NetFlow data, specifically in reference to nfcapd, nfsen? I have seen a wide range of implementations of NetFlow among various Net Flow-enabled devices. It would be great to know where the "keys" are, in order to make a good analysis.
Many thanks for reading and for any information you can provide! Also note: within reason, I am willing to pay for someone's time whoever can fix this. Of course the fix would be contributed back to here for Peter's review for inclusion in the next version.