pkt with new format was sent before new template is out

[pwp333]

We are seeing some invalid data got reported from time to time. The cause seems that flow pkt was sent before new template. We were wondering if iptnetflow only sends template at fix interval even when pkt using new template is sent out already.

The only difference between two templates seems to add only this new field.

        "uint8",
        "tcp_flags"

Would decreasing this value helps for this issue? refresh-rate=20

• for NetFlow v9 and IPFIX it's rate how frequently to re-send templates (per packets). You probably don't need to change default (which is 20).

Looking forward to your advice. Thanks.

[aabc]

Template should always be sent before first use. But, since this is UDP some packets may be lost.

[aabc]

See alloc_record_tpl:

                /* export template if needed */
                if (!tpl->exported_ts ||
                    pdu_count > (tpl->exported_cnt + refresh_rate) ||
                    time_is_before_jiffies(tpl->exported_ts + timeout_rate_j())) {
                        pdu_add_template(tpl);
                }

Condition is not only refresh rate, but if exported_ts is zero (it's never exported before) it would be exported too. This check happen always before data flow is added to the packet.

[aabc]

Template should always be sent before first use. But, since this is UDP some packets may be lost.

Also, one some network paths (devices) packet reorder could happen.

[pwp333]

Template should always be sent before first use. But, since this is UDP some packets may be lost.

!tpl->exported_ts may not be enough if template changes after it has been initial sent out. This is what we saw that the invalid data suddenly comes in after running netflow for a while. We need to send out template immediately after it has been changed.

[aabc]

A template does not change, all templates are unique.

[pwp333]

Is it reasonable to always send tcp_flags in template since this is the only issue we see when template change?

[aabc]

1. Please record by some traffic recorder such as Wireshark packets with the same template ID, but different template content (w/o restarting ipt-netflow of course).
2. If you want to experiment, you may remove TCP_FLAGS, line there and they will not be exported:

   3036 static struct base_template template_ports = {
   3037         .types = {
   3038                 L4_SRC_PORT,
   3039                 L4_DST_PORT,
   3040                 TCP_FLAGS,
   3041                 0
   3042         }
   3043 };

[pwp333]

Thanks fo the advice. It is very hard to capture the issue as it only happens once in a month. Is it possible to always export TCP_FLAGS so that template won't change?

[aabc]

As I said before, templates don't change. In the sense of assignment of template ID to template content. Unless ipt-netflow is restarted.

Also, note that the same unit contains TCP src/dst ports. Always, for any UDP or TCP packet there is "PORTs" AND "TCP_FLAGS" fields.

[pwp333]

We are lucky enough to get a bad example by reloading ipt-netflow. Please see the attached two json files for parsed template. template.zip

Please see template id "1007902103|259" and "1007902103|260". In both cases, tcp_flag was in one template but not the other.

        [
            "uint8",
            "tcp_flags"
        ],

1. The question is why tcp_flags can be missing in some template with same id? since you mentioned

   Always, for any UDP or TCP packet there is "PORTs" AND "TCP_FLAGS" fields. This template is for either TCP or UDP.

2. Also can we gen random template id upon each load of ipt-netflow

Thanks for your advice in advance.

[aabc]

Thanks, but json text does not authoritatively shows anything (for example why it itself is generated correctly?), nor I understand them. I can understand packet capture. Or fully parsed packet capture in a text form in a way wireshark can show them (tshark -P).

We are lucky enough to get a bad example by reloading ipt-netflow. Please see the attached two json files for parsed template.

Please see template id "1007902103|259" and "1007902103|260".

These numbers does not look like template ids. Also, they are different numbers, so these should be different templates.

In both cases, tcp_flag was in one template but not the other.

        [
            "uint8",
            "tcp_flags"
        ],

1. The question is why tcp_flags can be missing in some template with same id? since you mentioned

This is impossible, unless ipt-netflow is restarted between template exports.

Always, for any UDP or TCP packet there is "PORTs" AND "TCP_FLAGS" fields. This template is for either TCP or UDP.

1. Also can we gen random template id upon each load of ipt-netflow

This is possible. Recompile with ./configure --enable-rand-tpl-id see ./configure --help.

Thanks for your advice in advance.

[pwp333]

For this notation "1007902103|259", left part is hostid while right part is template id. We did restart ipt-netflow to repro the issue.

Looks this commit is exactly what we need here. Will give it a try. Thanks.

commit 56831a5bff2d22f5e5f90d6198fafb5a1abc46e0 Author: Simon Chopin s.chopin@alphalink.fr Date: Fri Jan 22 17:32:54 2021 +0100

ipt_netflow: optionally seed initial template ID from PRNG