search

openwrt / openwrt

This repository is a mirror of https://git.openwrt.org/openwrt/openwrt.git It is for reference only and is not active for check-ins. We will continue to accept Pull Requests here. They will be merged via staging trees then into openwrt.git.
Other
20.37k stars 10.52k forks source link

qmi_wwan skb_under_panic on IPv6 qmimux (QMAP) interface #16626

Open gsr933 opened 1 month ago

gsr933 commented 1 month ago

Describe the bug

Kernel panics with

skbuff: skb_under_panic: text:829b0fa4 len:64 put:4 head:8218aa00 data:8218a9fc tail:0x8218aa3c end:0x8218ab00 dev:qmimux0

when an incoming IPv6 TCP packet from a qmimux interface is rejected by the firewall. There is not enough headroom for the QMAP header in the TCP reset skb from netfilter.

OpenWrt version

r20134-5f15225c1e

OpenWrt release

22.03.5

OpenWrt target/subtarget

ramips/mt7621

Device

Mesh MK01K21

Image kind

Self-built image

Steps to reproduce

  1. Setup a qmimux interface with an IPv6 address
  2. From an outside host, connect to the address from (1) via TCP on a port that the firewall rejects (any port on the default firewall configuration)

Actual behaviour

Kernel panics with

skbuff: skb_under_panic: text:829b0fa4 len:64 put:4 head:8218aa00 data:8218a9fc tail:0x8218aa3c end:0x8218ab00 dev:qmimux0

Expected behaviour

The connection is rejected with a TCP reset

Additional info

net/ipv6/netfilter/nf_reject_ipv6.c : nf_send_reset6 uses dev->hard_header_len to reserve headroom. The existing qmimux_setup has dev->hard_header_len = 0. When the IPv6 TCP reset from netfilter is transmitted in qmimux_start_xmit, skb_push(skb, sizeof(struct qmimux_hdr)) fails with skb_under_panic because there is not enough headroom for the QMAP header.

I was able to fix this by setting hard_header_len and needed_headroom in qmimux_setup but I'm not sure if both are necessary.

static void qmimux_setup(struct net_device *dev)
{
    dev->header_ops      = NULL;  /* No header */
    dev->type            = ARPHRD_NONE;
    dev->hard_header_len = sizeof(struct qmimux_hdr);
    dev->needed_headroom = sizeof(struct qmimux_hdr);
    ...
}

The netfilter IPv4 TCP reset uses LL_MAX_HEADER so this should be not an issue.

Diffconfig

No response

Terms

brada4 commented 1 month ago

Do you have a backtrace? Identified code is from upstream, should fail same on any linux?

gsr933 commented 1 month ago

Do you have a backtrace? Identified code is from upstream, should fail same on any linux?

Full trace from serial console:

[44448.699396] skbuff: skb_under_panic: text:829b0fa4 len:64 put:4 head:8218aa00 data:8218a9fc tail:0x8218aa3c end:0x8218ab00 dev:qmimux0
[44448.711474] Kernel bug detected[#1]:
[44448.715038] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.10.176 #0
[44448.721097] $ 0   : 00000000 00000001 0000007a 003a2000
[44448.726312] $ 4   : 813db378 813db378 813e08f8 8180b880
[44448.731522] $ 8   : ffffefff 00000000 ffffffea 00000000
[44448.736732] $12   : 8180b88c 000001a8 80814920 ffffffff
[44448.741941] $16   : 82e0c000 81a7d300 0000003c 82f4cd00
[44448.747151] $20   : 80810000 00000000 8104049a 8103ecc8
[44448.752363] $24   : 00000018 803fbd8c                  
[44448.757574] $28   : 807fa000 8180ba38 80810000 804d4360
[44448.762784] Hi    : 0000016e
[44448.765644] Lo    : 2f688000
[44448.768521] epc   : 804d4360 skb_panic+0x5c/0x60
[44448.773114] ra    : 804d4360 skb_panic+0x5c/0x60
[44448.777710] Status: 11008403 KERNEL EXL IE 
[44448.781884] Cause : 50800024 (ExcCode 09)
[44448.785869] PrId  : 0001992f (MIPS 1004Kc)
[44448.789938] Modules linked in: qcserial pppoe ppp_async option nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet usb_wwan qmi_wwan pppox ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7915e mt76_connac_lib mt76 mac80211 cfg80211 vhci_hcd usbserial usbnet usbip_host usbip_core slhc nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_ipv6 nf_log_ipv4 nf_log_common nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c hwmon crc_ccitt compat cdc_wdm ledtrig_usbport nfsv4 nfsv3 nfs nfs_ssc nat46 oid_registry lockd sunrpc grace cifs dns_resolver sha512_generic sha256_generic libsha256 seqiv jitterentropy_rng drbg md5 md4 hmac ecb des_generic libdes cmac leds_gpio xhci_plat_hcd xhci_pci xhci_mtk xhci_hcd gpio_button_hotplug usbcore nls_base usb_common mii
[44448.790267]  crc32c_generic
[44448.879824] Process swapper/0 (pid: 0, threadinfo=657a9fe2, task=692e4d6f, tls=00000000)
[44448.887870] Stack : 00000000 807a5514 829b0fa4 00000040 00000004 8218aa00 8218a9fc 8218aa3c
[44448.896204]         8218ab00 82e0c000 00000000 804d43a8 00004000 00000000 82e0c000 8080c414
[44448.904537]         00000000 829b0fa4 d0000200 00000000 0130d019 00000000 81a7d300 82e0c000
[44448.912871]         00000000 804f7948 8080c414 804f71a4 81a7d300 8180bb5c 81a7d300 8080cd68
[44448.921206]         81a7d300 81040498 82e0c000 82f4cd00 8080c414 00000000 fffffff0 829e0000
[44448.929543]         ...
[44448.931981] Call Trace:
[44448.934417] [<804d4360>] skb_panic+0x5c/0x60
[44448.938667] [<804d43a8>] skb_send_sock_locked+0x0/0x238
[44448.943861] 
[44448.945338] Code: 01003825  0c020006  24845310 <000c000d> 8c8200a4  8c890054  8c8800a0  00451023  01254821 
[44448.955063] 
[44448.956801] ---[ end trace 6ab3ff584861859e ]---
[44448.961397] Kernel panic - not syncing: Fatal exception in interrupt
[44448.967735] ------------[ cut here ]------------
[44448.972372] WARNING: CPU: 0 PID: 0 at kernel/smp.c:633 smp_call_function_many_cond+0x438/0x454
[44448.980940] Modules linked in: qcserial pppoe ppp_async option nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet usb_wwan qmi_wwan pppox ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7915e mt76_connac_lib mt76 mac80211 cfg80211 vhci_hcd usbserial usbnet usbip_host usbip_core slhc nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_ipv6 nf_log_ipv4 nf_log_common nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c hwmon crc_ccitt compat cdc_wdm ledtrig_usbport nfsv4 nfsv3 nfs nfs_ssc nat46 oid_registry lockd sunrpc grace cifs dns_resolver sha512_generic sha256_generic libsha256 seqiv jitterentropy_rng drbg md5 md4 hmac ecb des_generic libdes cmac leds_gpio xhci_plat_hcd xhci_pci xhci_mtk xhci_hcd gpio_button_hotplug usbcore nls_base usb_common mii
[44448.981269]  crc32c_generic
[44449.070835] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G      D           5.10.176 #0
[44449.078281] Stack : 81170000 00000000 80810000 80083898 80880000 8077724c 00000000 00000000
[44449.086619]         8180b73c 81150000 8074578c 8080f330 8080ee47 00000001 8180b6e0 7f650cf9
[44449.094954]         00000000 00000000 8074578c 8180b580 ffffefff 00000000 ffffffea 00000000
[44449.103288]         8180b58c 000001cf 80814920 ffffffff 00000000 00000000 00000000 80740000
[44449.111623]         00000009 00000000 8104049a 00000000 00000018 803fbd8c 00000000 81150000
[44449.119959]         ...
[44449.122397] Call Trace:
[44449.124847] [<800080f0>] show_stack+0x30/0x100
[44449.129275] [<80379044>] dump_stack+0x9c/0xcc
[44449.133625] [<800301d4>] __warn+0xb0/0x11c
[44449.137701] [<8003029c>] warn_slowpath_fmt+0x5c/0xac
[44449.142650] [<800c1f60>] smp_call_function_many_cond+0x438/0x454
[44449.148626] [<800c1fc0>] smp_call_function+0x24/0x30
[44449.153567] [<8002fbf8>] panic+0x128/0x318
[44449.157642] [<800083b0>] die+0x10c/0x118
[44449.161543] [<800087ac>] do_trap_or_bp+0x150/0x1bc
[44449.166311] [<800088f0>] do_bp+0xd8/0x280
[44449.170299] [<80003e0c>] handle_bp_int+0x28/0x3c
[44449.174904] [<804d4360>] skb_panic+0x5c/0x60
[44449.179151] [<804d43a8>] skb_send_sock_locked+0x0/0x238
[44449.184346] 
[44449.185825] ---[ end trace 6ab3ff584861859f ]---
[44449.190435] Rebooting in 3 seconds..

Not quite usefull but I did some printf debugging and this is how it fails (comments mine):

hard_header_len initialized to 0:

static void qmimux_setup(struct net_device *dev)
{
    dev->header_ops      = NULL;
    dev->type            = ARPHRD_NONE;
    dev->hard_header_len = 0;
    ...
}

TCP reset skb from netfilter:

net/ipv6/netfilter/nf_reject_ipv6.c:
void nf_send_reset6(struct net *net, struct sk_buff *oldskb, int hook)
{
    ...
    hh_len = (dst->dev->hard_header_len + 15)&~15;    // hard_header_len == 0, hh_len == 0
    nskb = alloc_skb(hh_len + 15 + dst->header_len + sizeof(struct ipv6hdr)
             + sizeof(struct tcphdr) + dst->trailer_len,
             GFP_ATOMIC);
    ...
    skb_reserve(nskb, hh_len + dst->header_len);
    ...
}

skb transmit on qmi_wwan qmimux (qmap) interface:

drivers/net/usb/qmi_wwan.c:
static netdev_tx_t qmimux_start_xmit(struct sk_buff *skb, struct net_device *dev)
{
    struct qmimux_priv *priv = netdev_priv(dev);
    unsigned int len = skb->len;
    struct qmimux_hdr *hdr;
    netdev_tx_t ret;

    // zero headroom for this:
    hdr = skb_push(skb, sizeof(struct qmimux_hdr));
    ...
}

net/core/skbuff.c:
void *skb_push(struct sk_buff *skb, unsigned int len)
{
    skb->data -= len;
    skb->len  += len;
    if (unlikely(skb->data < skb->head))
        skb_under_panic(skb, len, __builtin_return_address(0));
    ...
}

This should fail on any Linux system under the same cirsumstances.

brada4 commented 1 month ago

Code you jighlighted is same in 23,05.5 and snapshot. Csn you try one of them?