search

CESNET / ipfixcol

IPFIXcol is an implementation of an IPFIX (RFC 7011) collector
Other
64 stars 37 forks source link

Template ID displayed incorrectly when using ipfixviewer #189

Closed halaney closed 7 years ago

halaney commented 7 years ago

This may or may not be a bug... I am just getting into using IPFIX so correct me if my understanding is incorrect. I am writing my own exporter and am using ipfixcol (v0.9.4 on ubuntu server 16.04) as the collector. Right now I am just exporting some arbitrary data, in this case a template with an ID of 1000 and a field for sourceTransportPort. When viewing with ipfixviewer on the collector the ID is 256 no matter what. Also I am sending a sourceTransportPort of 1, but when displayed in network byte order within ipfixviewer it appears in little-endian order. I captured the packets on both ends and they look correct in wireshark. I'm attaching the pcaps, the ipfix file, and a screenshot of the output from ipfixviewer. Let me know if I am misunderstanding something about templates or network byte order. ipfixcol_bug.zip

thorgrin commented 7 years ago

You are absolutely right here. The problem is that IPFIXcol rewrites template numbers to sequence. The reason for this is a historical one and I'd been racking my brain for the last hour to find out what it was. I think that we had been distinguishing sources by ODID in the past but when two exporters used the same ODID the templates collided. We use IP addresses and ports as well as ODIDs now, so the reson for renumbering the templates is most probably gone.

I'll have to do some more test on Monday, but I think that we can simply drop the renumbering, fix a few tests and all will be OK. However, do not send data from two exporters with the same ODID to the IPFIXcol when using the IPFIX file format storage. The format cannot keep track of real source and bad things will happen. I guess that we should add a big warning somewhere in this case.

Since you are using Ubuntu and not the RPM packages, I'll update this issue when we have committed a fix to the devel branch (it is currently the same as master since we did a new release just last week).

halaney commented 7 years ago

Awesome, thanks for being so quick to look into it. I appreciate the work you're doing, ipfixcol is pretty much the only collector I could find that can use TCP/SCTP. Did you notice my network order vs little-endian comment? The viewer is displaying "1" from the data record as "0x100" and claiming that is in network order... Wouldn't that be little-endian and not big-endian (aka network order)? You can see what I am talking about in the ipfixviewer_output.png file I included in the zip file.

thorgrin commented 7 years ago

Hi, I've read the other part of your question and then completely forgot about it due to the other issue. Anyway, I do not think that this is a bug. The port number is 1, and since it is a 16bit value, it is represented in the little endian as 0x0001. Converting to network order we get 0x0100 which can be written as 0x100, which is exactly what the ipfixviewer shows.

I've just committed the fix to the template issue to the devel branch, would you give it a try? If the issue persists, please reopen the issue. Also if you feel that you'd like to see the numbers printed by ipfixviewer in the little endian byte order, please open a new issue, or (preferrably ;-) ) a pull request.

halaney commented 7 years ago

Ohhhhh.... I was assuming the the first digit was the higher location in memory. I understand now, thanks. I'll go ahead and try out the devel branch

halaney commented 7 years ago

Just confirmed your fix works. Thanks again