Authenticator cannot observe the appended MAC

[h1994st]

This issue occurs on Linux 5.4.73 (Docker). The authenticator works as intended on Linux 5.4 (QEMU)

tcpdump results indicate that the authenticator network namespace actually receives the packet with the appended MAC. However, after some procedures (processing), the packet buffer passed to br_pass_frame_up or other ports has been changed.

Not sure:

• The appended MAC is excluded from the packet buffer?
• The tailer pointer of the packet buffer is changed? Why?

Linux Source Codes:

• https://elixir.bootlin.com/linux/v5.4/source/net/bridge/br_input.c
• https://elixir.bootlin.com/linux/v5.4.73/source/net/bridge/br_input.c

[h1994st]

• Try to compile Linux v5.4.73 and debug in QEMU
  • It still works correctly in QEMU
• View Diff: https://git.kernel.org/stable/ds/v5.4/v5.4.73

[h1994st]

Since the modules work well with Linux v5.4.73 in QEMU, another reason can be iptables+netfilter

• How to debug netfilters?

[h1994st]

Use netfilter hooks to debug (link):

[171888.260571] br_debug: NF_BR_PRE_ROUTING first
[171888.260573] br_debug: skb=ffff92fa2202bd00: len=64 headroom=16 head=ffff92fb867b9600 data=ffff92fb867b9610, tail=80, end=192
[171888.260573] br_debug: #hooks of NF_BR_PRE_ROUTING: 3
[171888.260580] br_debug: hook 0: br_pre_routing_first_hookfn+0x0/0xe0 [br_debug]
[171888.260582] br_debug: hook 1: br_nf_pre_routing+0x0/0x4e0 [br_netfilter]
[171888.260584] br_debug: hook 2: br_pre_routing_last_hookfn+0x0/0x70 [br_debug]
[171888.260587] br_debug: NF_INET_PRE_ROUTING first
[171888.260588] br_debug: skb=ffff92fa2202bd00: len=32 headroom=16 head=ffff92fb867b9600 data=ffff92fb867b9610, tail=48, end=192
[171888.260589] br_debug: #hooks of NF_INET_PRE_ROUTING: 3
[171888.260590] br_debug: hook 0: inet_pre_routing_first_hookfn+0x0/0xe0 [br_debug]
[171888.260592] br_debug: hook 1: ip_sabotage_in+0x0/0x60 [br_netfilter]
[171888.260593] br_debug: hook 2: inet_pre_routing_last_hookfn+0x0/0x70 [br_debug]
[171888.260594] br_debug: NF_INET_PRE_ROUTING last
[171888.260595] br_debug: skb=ffff92fa2202bd00: len=32 headroom=16 head=ffff92fb867b9600 data=ffff92fb867b9610, tail=48, end=192
[171888.260596] br_debug: NF_BR_PRE_ROUTING last
[171888.260597] br_debug: skb=ffff92fa2202bd00: len=32 headroom=16 head=ffff92fb867b9600 data=ffff92fb867b9610, tail=48, end=192
[171888.260609] br_debug: NF_BR_FORWARD
[171888.260610] br_debug: skb=ffff92fa2202a400: len=32 headroom=16 head=ffff92fb867b9600 data=ffff92fb867b9610, tail=48, end=192
[171888.260613] br_debug: NF_BR_POST_ROUTING
[171888.260614] br_debug: skb=ffff92fa2202a400: len=32 headroom=16 head=ffff92fb867b9600 data=ffff92fb867b9610, tail=48, end=192
[171888.260617] br_debug: NF_BR_LOCAL_IN
[171888.260618] br_debug: skb=ffff92fa2202bd00: len=32 headroom=16 head=ffff92fb867b9600 data=ffff92fb867b9610, tail=48, end=192

• There must exist other hooks that change the buffer tail
• Except for our own debug hooks, there is another hook function br_nf_pre_routing
• According to the log, br_nf_pre_routing or other hooks change the buffer tail
• Before entering inet_pre_routing_first_hookfn, the buffer tail has already been changed. Therefore, we should dig into br_nf_pre_routing

[h1994st]

In our host machine, CONFIG_BRIDGE_NETFILTER is enabled, so br_nf_pre_routing will be called.

In br_nf_pre_routing, br_validate_ipv4 will be used check the IP packet, and pskb_trim_rcsum is further invoked

static inline int pskb_trim_rcsum(struct sk_buff *skb, unsigned int len)
{
    if (likely(len >= skb->len))
        return 0;
    return pskb_trim_rcsum_slow(skb, len);
}

However, len here is equal to ntohs(iph->tot_len) (in br_validate_ipv4), while skb->len is larger than that, because the sender appends a MAC to the end of the packet. Thus, in __pskb_trim, skb->len and skb->tail will be reset to exclude trailing date.

[h1994st]

To resolve this issue, we can add netfilter hooks in the mitm kernel module to change the packet buffer length back.

[h1994st]

We change skb->len once after we receive the packet from the sender. At the receiver side, tcpdump can observe the broadcast UDP packet with the appended MAC, but the mitm_recv module cannot.