Ipfixcol filter and aggregation problems.

[zhexiao]

Dears,

I have few questions about the Ipfixcol filter and aggregation mechanism.

1. How to configure the Ipfixcol templates and where can i find the templates path?

2. How to use these Ipfixcol filter and aggregation plugins on my project?

3. I have a warning error occurred during the Ipfixcol start, could you please tell me what's the error meaning or the ID 256 meaning?

Jul 19 09:23:01 netflow-kafka ipfixcol[4492]: WARNING: preprocessor: [0] Data template with ID 256 has expired; using old template...
Jul 19 09:23:01 netflow-kafka ipfixcol[4492]: WARNING: preprocessor: [0] Sequence number error; expected 181820, got 181830
Jul 19 09:23:01 netflow-kafka ipfixcol[4492]: WARNING: preprocessor: [0] Data template with ID 256 not found

4. what's the "ipfix.protocolIdentifier": 6, meaning ? Where can find the exactly docs about these properties? For example:

   {
   "ipfix.protocolIdentifier": 6,  (Is "6" meaning TCP?)
   "ipfix.tcpControlBits": 27,
   "ipfix.egressInterface": 0,
   "ipfix.octetDeltaCount": 977,
   ...
   }

Thanks

Best Zhe

[Lukas955]

1) You cannot configure the templates because based on the IPFIX protocol an exporter is responsible for their definiton for each transport session between the exporter and a collector. Therefore, templates may vary for each exporter, or even session. If you want to manipulate with records that the collector receives, you should try to look into a configuration of your exporter if you can change exported Information Elements.

2) Almost every plugin has a manual page. For example, manual page of the filter plugin is called "ipfixcol-filter-inter". To find all available manual pages that are distributed with the collector and plugins on your computer run shell command $ apropos ipfixcol.

3) I guess you use the UDP input plugin. Because the IPFIX protocol is "one-way" push protocol and the UDP protocol doesn't allow the collector to easily signalize start of communication, the collector is not able to interpret flow records until the connected exporter will send all templates definitions. These definitions are usually send periodically based on your exporter configuration - look for parameters such as "templateRefreshTimeout" etc. In other words, during the collector start you can expect these warnings and the situation can last up to "templateRefreshTimeout" seconds. By the way, the sequence number errors are directly connected with this issue because the sequence number represents number of data records sent since start of the exporter and because the collector cannot interpret all data records, it is also not able to calculate expected sequence numbers of future IPFIX packets.

4) If you are interested in definitions of IPFIX Information Elements, such as "protocolIdentifier", you can find them here. In this case, you are right - 6 means TCP as you can see here. If you use JSON storage plugin, you are maybe interested in the parameter <protocol>formatted</protocol> that allows you to automatically convert raw numbers to human readable strings.

[zhexiao]

Dear Lukas,

I appreciated your answers. It's solve me a lot of problems, hope you have a great time.

Best Zhe