openwrt / mt76

mac80211 driver for MediaTek MT76x0e, MT76x2e, MT7603, MT7615, MT7628 and MT7688
731 stars 334 forks source link

Mt7603 Kernel panic. #193

Closed pparent76 closed 5 years ago

pparent76 commented 5 years ago

I ran a long time test on a WG3526 router with mt7603 driver enabled (and 5gz disable). It runned hostapd 2018-05-21-62566bc2-4 (not wpad-mini) and kmod-mt76 - 4.14.44+2018-05-20-792dbe02-1 ( commit 792dbe02613299f67e92815f0b2b7584dc951ffd ) and kernel - 4.14.44-1-8b1a603c6fa85e9d05d21ea6946e47fd

It did 20 unexpected crash in 72 days. Il does not happen with the wifi disabled.

Example of crashlog:

<1>[79032.004078] CPU 1 Unable to handle kernel paging request at virtual address 07407800, epc == 80108880, ra == 80108764
<4>[79032.014717] Oops[#1]:
<4>[79032.017009] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.44 #0
<4>[79032.023010] task: 8fc43e80 task.stack: 8fc64000
<4>[79032.027541] $ 0   : 00000000 00000001 00000000 814a56f0
<4>[79032.032780] $ 4   : 8058e1b8 00000001 00000001 07407800
<4>[79032.038020] $ 8   : 000c6a65 000c6a64 8f0bc830 8f3e8000
<4>[79032.043263] $12   : 80590000 ffffffff 8e21c930 00000000
<4>[79032.048506] $16   : 8fc02a00 01090220 8c89a000 8032f4d8
<4>[79032.053759] $20   : 00000800 8fc1e05c 01080020 01080020
<4>[79032.059005] $24   : 0000000a 8038af24                  
<4>[79032.064271] $28   : 8fc64000 8fc0dbc8 8058e8f0 80108764
<4>[79032.069525] Hi    : 00001ef9
<4>[79032.072422] Lo    : f704a000
<4>[79032.075348] epc   : 80108880 __kmalloc_track_caller+0x1d4/0x228
<4>[79032.081294] ra    : 80108764 __kmalloc_track_caller+0xb8/0x228
<4>[79032.087127] Status: 11007c03      KERNEL EXL IE 
<4>[79032.091316] Cause : 40800008 (ExcCode 02)
<4>[79032.095307] BadVA : 07407800
<4>[79032.098176] PrId  : 0001992f (MIPS 1004Kc)
<4>[79032.102257] Modules linked in: xt_coova rtl8192cu rtl8192c_common rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter em_cmp sch_multiq sch_teql em_nbyte sch_pie
<4>[79032.173110]  sch_gred sch_dsmark act_police cls_basic act_ipt sch_prio em_meta em_text sch_codel sch_sfq sch_fq sch_red act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug
<4>[79032.243700]  ext4 jbd2 mbcache usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[79032.252751] Process swapper/1 (pid: 0, threadinfo=8fc64000, task=8fc43e80, tls=00000000)
<4>[79032.260805] Stack : 00000000 00000001 805ee694 8f11e5a0 8f5ec840 00000000 01080020 8032d5b0
<4>[79032.269150]         8f2bc000 8f11e5e0 8f11e638 00000014 8f5ec840 8d9c1b00 00000000 00000740
<4>[79032.277496]         000005dc 8032f4d8 8e9f9918 8f2bcd48 8f5ec840 ac07f535 8f5ec840 8f5ec840
<4>[79032.285839]         8d9c1b00 805af320 805b0000 000005dc 8fc1e05c 01080020 00000000 8038b22c
<4>[79032.347627]         00000004 803815d4 8e75a064 8fc0dca4 8f180b80 800657f0 80590002 8fc1e000
<4>[79032.355978]         ...
<4>[79032.358425] Call Trace:
<4>[79032.360900] [<80108880>] __kmalloc_track_caller+0x1d4/0x228
<4>[79032.366488] [<8032d5b0>] __kmalloc_reserve.isra.7+0x40/0xa4
<4>[79032.372053] [<8032f4d8>] pskb_expand_head+0x8c/0x31c
<4>[79032.377024] [<8038b22c>] ip_forward+0x308/0x48c
<4>[79032.381538] [<80389ce0>] ip_rcv+0x2a0/0x320
<4>[79032.385719] [<80340078>] __netif_receive_skb_core+0xa94/0xc5c
<4>[79032.391463] [<80345fe8>] netif_receive_skb_internal+0xd8/0xf0
<4>[79032.397186] [<8044ba70>] br_pass_frame_up+0xe8/0x154
<4>[79032.402136] [<8044c070>] br_handle_frame_finish+0x53c/0x580
<4>[79032.407683] [<8044c3e4>] br_handle_frame+0x330/0x3dc
<4>[79032.412630] [<8033fd6c>] __netif_receive_skb_core+0x788/0xc5c
<4>[79032.418387] [<80342be4>] process_backlog+0x98/0x160
<4>[79032.423258] [<80346454>] net_rx_action+0x150/0x30c
<4>[79032.428054] [<8047cb90>] __do_softirq+0x128/0x2ec
<4>[79032.432777] [<80032910>] irq_exit+0x98/0xcc
<4>[79032.436972] [<8024a6cc>] plat_irq_dispatch+0xfc/0x138
<4>[79032.442012] [<8000b5a8>] except_vec_vi_end+0xb8/0xc4
<4>[79032.446958] [<8000cf70>] r4k_wait_irqoff+0x1c/0x24
<4>[79032.451757] [<8006645c>] do_idle+0xe4/0x168
<4>[79032.455934] [<800666d8>] cpu_startup_entry+0x24/0x2c
<4>[79032.460884] Code: 00000000  8e020014  00e23821 <8ce20000> 10000009  cc400000  1040ffbd  00000000  8e060010 
<4>[79032.470617] 
<4>[79032.472480] ---[ end trace 8ecf7d1623c6d529 ]---

Hostapd configuration file :

cat /etc/hostapd.conf 
driver=nl80211
country_code=00
ieee80211d=1
hw_mode=g
beacon_int=100

noscan=1
ieee80211n=1
ht_coex=0
ht_capab=[HT40+][SHORT-GI-20][SHORT-GI-40][TX-STBC][RX-STBC1]

ctrl_interface=/var/run/hostapd
disassoc_low_ack=1
preamble=1
wmm_enabled=1
uapsd_advertisement_enabled=1

interface=wlan0
channel=1
ssid=IXXXXXXXXX
bridge=br-lan
ap_isolate=1
easyteacher commented 5 years ago

Same kernel oops occurred on kernel 4.14.62.

pparent76 commented 5 years ago

I'm running a new test with commit fe650dab5cd824c592bc40ecdb512176966b4293 and kernel 4.14.68, I'll tell you in few days/weeks if this kernel panic still happens.

pparent76 commented 5 years ago

One crash in 7 days of testing on commit fe650dab5cd824c592bc40ecdb512176966b4293, I will try with newer commits.

<1>[378825.496023] CPU 2 Unable to handle kernel paging request at virtual address 07406000, epc == 80108d38, ra == 80108c1c
<4>[378825.506704] Oops[#1]:
<4>[378825.509057] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.14.68 #0
<4>[378825.515121] task: 8fc444c0 task.stack: 8fc66000
<4>[378825.519713] $ 0   : 00000000 00000001 00000000 814b36f0
<4>[378825.525011] $ 4   : 805a01b8 00000001 00000001 07406000
<4>[378825.530306] $ 8   : 002e66b2 002e66b1 8f0d4830 8f31a000
<4>[378825.535603] $12   : 805a0000 ffffffff 8e38d130 00000000
<4>[378825.540899] $16   : 8fc02a00 01090220 8e5f3000 8033b1d4
<4>[378825.546196] $20   : 00000800 8e71405c 01080020 01080020
<4>[378825.551492] $24   : 0000000a 80396ee4                  
<4>[378825.556789] $28   : 8fc66000 8fc11bc8 805a08f0 80108c1c
<4>[378825.562088] Hi    : 00000fc1
<4>[378825.565037] Lo    : 7d684000
<4>[378825.568018] epc   : 80108d38 __kmalloc_track_caller+0x1d4/0x228
<4>[378825.573998] ra    : 80108c1c __kmalloc_track_caller+0xb8/0x228
<4>[378825.579885] Status: 11007c03     KERNEL EXL IE 
<4>[378825.584146] Cause : 40800008 (ExcCode 02)
<4>[378825.588219] BadVA : 07406000
<4>[378825.591167] PrId  : 0001992f (MIPS 1004Kc)
<4>[378825.595326] Modules linked in: xt_coova rtl8192cu rtl8192c_common rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter sch_multiq sch_teql em_nbyte
<4>[378825.665859]  sch_prio sch_dsmark sch_pie act_ipt sch_gred em_meta cls_basic em_cmp act_police em_text sch_codel sch_fq sch_sfq sch_red act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug
<4>[378825.737829]  ext4 jbd2 mbcache usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[378825.746950] Process swapper/2 (pid: 0, threadinfo=8fc66000, task=8fc444c0, tls=00000000)
<4>[378825.755084] Stack : 00000000 00000001 80600694 8f11e5b0 8efa5600 00000000 01080020 803392f4
<4>[378825.763506]         8e0ac000 8f11e5f0 8f11e648 00000014 8efa5600 8e07a180 00000000 00000740
<4>[378825.771937]         000005dc 8033b1d4 8f155918 8e0acd48 8efa5600 ac07f55d 8efa5600 8efa5600
<4>[378825.780359]         8e07a180 805c1760 805c0000 000005dc 8e71405c 01080020 00000000 803971fc
<4>[378825.788776]         00000005 8038d454 8d3da864 8fc11ca4 8fd3e900 80065aa0 805a0000 803335a0
<4>[378825.797196]         ...
<4>[378825.799719] Call Trace:
<4>[378825.802261] [<80108d38>] __kmalloc_track_caller+0x1d4/0x228
<4>[378825.807910] [<803392f4>] __kmalloc_reserve.isra.7+0x40/0xa4
<4>[378825.813552] [<8033b1d4>] pskb_expand_head+0x88/0x318
<4>[378825.818603] [<803971fc>] ip_forward+0x318/0x49c
<4>[378825.823201] [<80395c80>] ip_rcv+0x2a0/0x320
<4>[378825.827482] [<8034be58>] __netif_receive_skb_core+0xa94/0xc5c
<4>[378825.833308] [<80351dc0>] netif_receive_skb_internal+0xd8/0xf0
<4>[378825.839139] [<80458010>] br_pass_frame_up+0xe8/0x154
<4>[378825.844172] [<80458608>] br_handle_frame_finish+0x534/0x578
<4>[378825.849805] [<8045897c>] br_handle_frame+0x330/0x3d4
<4>[378825.854835] [<8034bb4c>] __netif_receive_skb_core+0x788/0xc5c
<4>[378825.860640] [<8034e9c4>] process_backlog+0x98/0x160
<4>[378825.865585] [<8035222c>] net_rx_action+0x150/0x30c
<4>[378825.870460] [<80489410>] __do_softirq+0x128/0x2ec
<4>[378825.875245] [<80032ab4>] irq_exit+0xac/0xc8
<4>[378825.879518] [<8024b27c>] plat_irq_dispatch+0xfc/0x138
<4>[378825.884644] [<8000b5e8>] except_vec_vi_end+0xb8/0xc4
<4>[378825.889675] [<8000cfb0>] r4k_wait_irqoff+0x1c/0x24
<4>[378825.894557] [<8006670c>] do_idle+0xe4/0x168
<4>[378825.898808] [<80066988>] cpu_startup_entry+0x24/0x2c
<4>[378825.903850] [<800180b4>] not_nmi+0x8c/0x1d8
<4>[378825.908101] Code: 00000000  8e020014  00e23821 <8ce20000> 10000009  cc400000  1040ffbd  00000000  8e060010 
<4>[378825.917914] 
<4>[378825.919775] ---[ end trace bba04014ab229012 ]---
pparent76 commented 5 years ago

So overall I had 3 crashes in 14 days on commit fe650da . I start testing commit 6e1898d60a780c9d89dff9cbb3569f267db13e21 today. I'll tell you if there are still crashes or not.

pparent76 commented 5 years ago

New crash with commit 6e1898d

<1>[420536.263279] CPU 1 Unable to handle kernel paging request at virtual address 07406000, epc == 80108e38, ra == 80108d1c
<4>[420536.273986] Oops[#1]:
<4>[420536.276355] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.71 #0
<4>[420536.282431] task: 8fc43e80 task.stack: 8fc64000
<4>[420536.287035] $ 0   : 00000000 00000001 00000000 814a56f0
<4>[420536.292339] $ 4   : 805a01b8 00000001 00000001 07406000
<4>[420536.297654] $ 8   : 003f8ea9 003f8ea8 8f0cc830 8e446000
<4>[420536.302970] $12   : 805a0000 ffffffff 8db61130 00000000
<4>[420536.308274] $16   : 8fc02a00 01090220 8df04800 8033b5c4
<4>[420536.313585] $20   : 00000800 8fd5505c 01080020 01080020
<4>[420536.318889] $24   : 0000000a 803974c4                  
<4>[420536.324188] $28   : 8fc64000 8fc0dbc8 805a08f0 80108d1c
<4>[420536.329506] Hi    : 0000017c
<4>[420536.332467] Lo    : 74756000
<4>[420536.335461] epc   : 80108e38 __kmalloc_track_caller+0x1d4/0x228
<4>[420536.341473] ra    : 80108d1c __kmalloc_track_caller+0xb8/0x228
<4>[420536.347389] Status: 11007c03     KERNEL EXL IE 
<4>[420536.351673] Cause : 40800008 (ExcCode 02)
<4>[420536.355761] BadVA : 07406000
<4>[420536.358726] PrId  : 0001992f (MIPS 1004Kc)
<4>[420536.362911] Modules linked in: xt_coova rtl8192cu rtl8192c_common rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter sch_multiq sch_teql em_nbyte
<4>[420536.433660]  sch_dsmark sch_pie sch_gred em_meta cls_basic em_text act_ipt sch_codel sch_sfq act_police sch_fq sch_prio em_cmp sch_red act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug
<4>[420536.505703]  ext4 jbd2 mbcache usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[420536.514881] Process swapper/1 (pid: 0, threadinfo=8fc64000, task=8fc43e80, tls=00000000)
<4>[420536.523046] Stack : 00000000 00000001 80600694 8f1725b0 8f1a69c0 00000000 01080020 803396e4
<4>[420536.531512]         8e770000 8f1725f0 8f172648 00000014 8f1a69c0 8d867b00 00000000 00000740
<4>[420536.539977]         000005dc 8033b5c4 8f16f918 8e770d48 8f1a69c0 ac07f55e 8f1a69c0 8f1a69c0
<4>[420536.548465]         8d867b00 805c1760 805c0000 000005dc 8fd5505c 01080020 00000000 803977dc
<4>[420536.556918]         00000005 8038d910 8f2b7864 8fc0dca4 8f1d2180 80065bf0 805a0802 8fd55000
<4>[420536.565378]         ...
<4>[420536.567936] Call Trace:
<4>[420536.570510] [<80108e38>] __kmalloc_track_caller+0x1d4/0x228
<4>[420536.576225] [<803396e4>] __kmalloc_reserve.isra.7+0x40/0xa4
<4>[420536.581921] [<8033b5c4>] pskb_expand_head+0x88/0x318
<4>[420536.587037] [<803977dc>] ip_forward+0x318/0x49c
<4>[420536.591676] [<803961f0>] ip_rcv+0x260/0x2e0
<4>[420536.595958] [<8034c2c8>] __netif_receive_skb_core+0xa94/0xc5c
<4>[420536.601821] [<80352230>] netif_receive_skb_internal+0xd8/0xf0
<4>[420536.607709] [<80458360>] br_pass_frame_up+0xe8/0x154
<4>[420536.612822] [<80458958>] br_handle_frame_finish+0x534/0x578
<4>[420536.618529] [<80458ccc>] br_handle_frame+0x330/0x3d4
<4>[420536.623600] [<8034bfbc>] __netif_receive_skb_core+0x788/0xc5c
<4>[420536.629455] [<8034ee34>] process_backlog+0x98/0x160
<4>[420536.634438] [<8035269c>] net_rx_action+0x150/0x30c
<4>[420536.639368] [<80489740>] __do_softirq+0x128/0x2ec
<4>[420536.644196] [<80032b34>] irq_exit+0xac/0xc8
<4>[420536.648501] [<8024b3fc>] plat_irq_dispatch+0xfc/0x138
<4>[420536.653660] [<8000b5e8>] except_vec_vi_end+0xb8/0xc4
<4>[420536.658709] [<8000cfb0>] r4k_wait_irqoff+0x1c/0x24
<4>[420536.663609] [<8006685c>] do_idle+0xe4/0x168
<4>[420536.667875] [<80066ad8>] cpu_startup_entry+0x24/0x2c
<4>[420536.672921] Code: 00000000  8e020014  00e23821 <8ce20000> 10000009  cc400000  1040ffbd  00000000  8e060010 
<4>[420536.682742] 
<4>[420536.684645] ---[ end trace 5ab0c9a2bdd404b0 ]---

===================================
Time: 1538306264.122814
nbd168 commented 5 years ago

I just pushed a crash fix. Please try the latest OpenWrt version (18.06 or master branch)

pparent76 commented 5 years ago

Thank's a lot! If it works it would be amazing!

I upgrade my router on Monday (I cannot do it Before). And I tell you in few days or weeks if the problem is solved!

pparent76 commented 5 years ago

Ok I've started to test commit d533f1351834ba2899897b090aa56a5f5b922bd7 on latest trunk version! I'll tell you in 7 days if it is still up or before if there is a crash

camelcamro commented 5 years ago

still having kernel panics with crash/reboot (WG3526) -> newest trunk) 1-2 per day ... kernel paging request at virtual address is that related to 5GHZ or 2,4 Ghz wireless stuff ? should i try to disable the 5Ghz and testing again ?

nbd168 commented 5 years ago

Please show me the full crash log

pparent76 commented 5 years ago

should i try to disable the 5Ghz and testing again ?

I'm always testing with 5Ghz disabled for now.

No crash yet on my side but it's only been 48H of testing.

pparent76 commented 5 years ago

New crash log:

<1>[221549.254676] CPU 1 Unable to handle kernel paging request at virtual address 07406000, epc == 8010c560, ra == 8010c444
<4>[221549.265366] Oops[#1]:
<4>[221549.267719] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.75 #0
<4>[221549.273788] task: 8fc43e80 task.stack: 8fc64000
<4>[221549.278381] $ 0   : 00000000 00000001 00000000 814a56f0
<4>[221549.283681] $ 4   : 805a61b8 00000001 00000001 07406000
<4>[221549.288978] $ 8   : 002faae5 002faae4 00000000 00000000
<4>[221549.294276] $12   : 00000000 8f3604b8 00000001 00000002
<4>[221549.299572] $16   : 8fc02a00 01090220 8e17a000 803413c0
<4>[221549.304871] $20   : 00000720 8e9fcbc0 8e9fcbc0 01080020
<4>[221549.310170] $24   : 00000005 00000003                  
<4>[221549.315469] $28   : 8fc64000 8fc0d990 8fe86054 8010c444
<4>[221549.320768] Hi    : 0000362e
<4>[221549.323719] Lo    : 000001aa
<4>[221549.326707] epc   : 8010c560 __kmalloc_track_caller+0x1d4/0x228
<4>[221549.332687] ra    : 8010c444 __kmalloc_track_caller+0xb8/0x228
<4>[221549.338578] Status: 11007c03     KERNEL EXL IE 
<4>[221549.342843] Cause : 40800008 (ExcCode 02)
<4>[221549.346915] BadVA : 07406000
<4>[221549.349864] PrId  : 0001992f (MIPS 1004Kc)
<4>[221549.354022] Modules linked in: xt_coova rtl8192cu rtl8192c_common rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter sch_multiq sch_teql em_nbyte
<4>[221549.424523]  sch_codel sch_prio sch_dsmark act_ipt sch_gred em_meta cls_basic em_text sch_pie sch_fq sch_sfq act_police em_cmp sch_red act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug
<4>[221549.496483]  ext4 jbd2 mbcache usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[221549.505609] Process swapper/1 (pid: 0, threadinfo=8fc64000, task=8fc43e80, tls=00000000)
<4>[221549.513746] Stack : 00000856 00000168 00000001 00000000 8ef079c0 00000000 01080020 8033f4e8
<4>[221549.522172]         811c5f20 8ff1b538 00003fff 803110e4 8ef079c0 8fe86480 00000000 00000660
<4>[221549.530594]         8d98e500 803413c0 8ef07a1c 8fe86480 8f13c240 8f305900 00000042 8e9fcbc0
<4>[221549.539014]         8fe86480 8fe86000 8eda9800 8d98e500 8e9fcbc0 8e9fcbc0 00000014 8f32dca8
<4>[221549.547439]         8ef07a80 8ef079c0 8fe86480 8fe86000 8ef079c0 8f331f20 805a68f0 ffffffff
<4>[221549.555864]         ...
<4>[221549.558388] Call Trace:
<4>[221549.560934] [<8010c560>] __kmalloc_track_caller+0x1d4/0x228
<4>[221549.566598] [<8033f4e8>] __kmalloc_reserve.isra.7+0x40/0xa4
<4>[221549.572242] [<803413c0>] pskb_expand_head+0x88/0x318
<4>[221549.577415] [<8f32dca8>] ieee80211_beacon_get_tim+0x244/0xebc [mac80211]
<4>[221549.584214] Code: 00000000  8e020014  00e23821 <8ce20000> 10000009  cc400000  1040ffbd  00000000  8e060010 
<4>[221549.594032] 
<4>[221549.595903] ---[ end trace 1c49bbc839392b42 ]---
pparent76 commented 5 years ago

There was a total of 3 unexpected crashes in 8 days. Another crash:

<1>[24835.420249] CPU 1 Unable to handle kernel paging request at virtual address 07403fe0, epc == 800fc214, ra == 800fc374
<4>[24835.430942] Oops[#1]:
<4>[24835.433211] CPU: 1 PID: 18812 Comm: iptables Not tainted 4.14.75 #0
<4>[24835.439451] task: 8fe53e80 task.stack: 8e8c6000
<4>[24835.443959] $ 0   : 00000000 00000001 07403fe0 811ca900
<4>[24835.449171] $ 4   : 03186000 c6352000 07406000 00000000
<4>[24835.454382] $ 8   : 00000000 ffffffe0 6e61776d 6f635f33
<4>[24835.459592] $12   : 63656e6e 7fb9bc28 00000000 00646574
<4>[24835.464803] $16   : 8e68c200 80640000 00000001 00000000
<4>[24835.470013] $20   : 8f0fec80 77f4d000 77f4d000 77f52ddc
<4>[24835.475225] $24   : 00000000 803cb81c                  
<4>[24835.480441] $28   : 8e8c6000 8e8c7dd0 00000000 800fc374
<4>[24835.485659] Hi    : 00004e56
<4>[24835.488523] Lo    : 0000049f
<4>[24835.491405] epc   : 800fc214 __purge_vmap_area_lazy+0x60/0x13c
<4>[24835.497217] ra    : 800fc374 free_vmap_area_noflush+0x84/0xa4
<4>[24835.502935] Status: 11007c03      KERNEL EXL IE 
<4>[24835.507106] Cause : 40800008 (ExcCode 02)
<4>[24835.511093] BadVA : 07403fe0
<4>[24835.513957] PrId  : 0001992f (MIPS 1004Kc)
<4>[24835.518027] Modules linked in: xt_coova rtl8192cu rtl8192c_common rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter sch_multiq sch_teql em_nbyte
<4>[24835.588436]  sch_codel sch_prio sch_dsmark act_ipt sch_gred em_meta cls_basic em_text sch_pie sch_fq sch_sfq act_police em_cmp sch_red act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug
<4>[24835.660311]  ext4 jbd2 mbcache usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[24835.669347] Process iptables (pid: 18812, threadinfo=8e8c6000, task=8fe53e80, tls=77f59d98)
<4>[24835.677656] Stack : ffffffff 8ec1dd80 00000001 800fe9f8 8e8c7f08 8e68c220 805b0000 80640000
<4>[24835.685997]         8e68cf80 00000000 8f0fec80 77f4d000 77f4d000 800fc374 8f0fec80 77f4d000
<4>[24835.694337]         77f4d000 77f52ddc 8e68c200 800fe3bc 77f4d000 77f4d000 77f52ddc 800fed68
<4>[24835.702678]         c6350000 8e68cf80 00000001 800fe450 014000c0 00000bcf 00000000 ffffffff
<4>[24835.711017]         c6350000 8e8c7ed4 7fb9bc2c 8f0f9b20 805a0000 00000000 8fc33000 8fda5f80
<4>[24835.719354]         ...
<4>[24835.721798] Call Trace:
<4>[24835.724244] [<800fc214>] __purge_vmap_area_lazy+0x60/0x13c
<4>[24835.729708] [<800fc374>] free_vmap_area_noflush+0x84/0xa4
<4>[24835.735090] [<800fe3bc>] remove_vm_area+0x80/0xa8
<4>[24835.739774] [<800fe450>] __vunmap+0x6c/0xf4
<4>[24835.744016] [<8f0f9b20>] ip_set_nfnl_put+0x3ac/0x212c [ip_set]
<4>[24835.749831] Code: 00009025  1000000a  2409ffe0 <8c460000> 8c430004  8c420020  00c4402b  00a3382b  00c8200b 
<4>[24835.759561] 
<4>[24835.761385] ---[ end trace 7ef5625a3d2214df ]---

===================================
Time: 1540281251.574381
<4>[24835.759561] 
<4>[24835.761385] ---[ end trace 7ef5625a3d2214df ]---
<0>[24835.771192] Kernel panic - not syncing: Fatal exception
nbd168 commented 5 years ago

Please try the latest version (master only, not 18.06).

pparent76 commented 5 years ago

For those would would like to know how to get crashlog after reboot:

cat /sys/kernel/debug/crashlog

ptpt52 commented 5 years ago

My two friends reported crash, the code is the latest master HEAD. They can't provide crash log, the devices are mir3g and r6220, both are mt76 drivers.

ptpt52 commented 5 years ago

May be related to lan transmission of broadcast packets, from wired lan to wireless transmission

ptpt52 commented 5 years ago

Will it be that some special packet in tx/rx path enters a recursive dead loop causing a stack overflow?

ybbid commented 5 years ago

For those would would like to know how to get crashlog after reboot:

cat /sys/kernel/debug/crashlog

thanks @pparent76 . is this need enable debug when compile openwrt firmware?

lukasz1992 commented 5 years ago

@ybbid Check if you have CONFIG_CRASHLOG kernel flag enabled.

pparent76 commented 5 years ago

I just updated my router with version "kmod-mt7603 - 4.14.78+2018-10-27-fdc63f16-2" I'm telling you in few days if there are still crashes.

ybbid commented 5 years ago

@ybbid Check if you have CONFIG_CRASHLOG kernel flag enabled.

@lukasz1992 thank you much. I'll have a try.

pparent76 commented 5 years ago

New crash on version "kmod-mt7603 - 4.14.78+2018-10-27-fdc63f16-2" (after almost 6 days up) :

<1>[502718.569848] CPU 3 Unable to handle kernel paging request at virtual address 07406000, epc == 8010b2b4, ra == 8002c568
<4>[502718.580639] Oops[#1]:
<4>[502718.583011] CPU: 3 PID: 23495 Comm: sh Not tainted 4.14.78 #0
<4>[502718.588821] task: 8dcc92c0 task.stack: 8cc04000
<4>[502718.593417] $ 0   : 00000000 00000001 00000000 814c22d8
<4>[502718.598729] $ 4   : 805a61b8 00000001 00000001 07406000
<4>[502718.604037] $ 8   : 0028c1bd 0028c1bc 00000000 00000000
<4>[502718.609346] $12   : 00000000 0089543b 77edaa2c 00000000
<4>[502718.614653] $16   : 8fc02100 014080c0 8e3f0000 8feaf954
<4>[502718.619962] $20   : 00000000 8e73c8c0 00000000 00000000
<4>[502718.625268] $24   : 00000000 77f15420                  
<4>[502718.630575] $28   : 8cc04000 8cc05e10 00000000 8002c568
<4>[502718.635884] Hi    : 0089543b
<4>[502718.638836] Lo    : b55e0000
<4>[502718.641830] epc   : 8010b2b4 kmem_cache_alloc+0x128/0x17c
<4>[502718.647300] ra    : 8002c568 copy_process.part.9+0x5ec/0x151c
<4>[502718.653105] Status: 11007c03     KERNEL EXL IE 
<4>[502718.657371] Cause : 40800008 (ExcCode 02)
<4>[502718.661450] BadVA : 07406000
<4>[502718.664405] PrId  : 0001992f (MIPS 1004Kc)
<4>[502718.668565] Modules linked in: xt_coova rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter em_cmp sch_multiq sch_teql em_nbyte sch_pie sch_gred sch_dsmark
<4>[502718.739969]  act_police cls_basic act_ipt sch_prio em_meta em_text sch_codel sch_sfq sch_fq sch_red act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug ext4 jbd2 mbcache
<4>[502718.810572]  usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[502718.818139] Process sh (pid: 23495, threadinfo=8cc04000, task=8dcc92c0, tls=77f5ed98)
<4>[502718.826024] Stack : 80610000 8feaf954 00000000 00000000 8feaf6c0 00000012 80610000 8002c568
<4>[502718.834474]         814393e0 00000000 8e723dfc 8006b378 00000055 7fd2a0b0 8dd49180 80610000
<4>[502718.870146]         00000055 7fd2a0b0 8dd49180 8001c528 00000000 8e747d00 8cc05ed8 77f57ddc
<4>[502718.878588]         00000000 8003a5d4 00000000 8e747d00 8cc05ed8 00000002 fffffff4 8005400c
<4>[502718.887032]         00000012 00000000 00000002 00430215 00000000 77f51000 77f51000 77f57ddc
<4>[502718.895485]         ...
<4>[502718.898014] Call Trace:
<4>[502718.900573] [<8010b2b4>] kmem_cache_alloc+0x128/0x17c
<4>[502718.905704] [<8002c568>] copy_process.part.9+0x5ec/0x151c
<4>[502718.911173] [<8002d62c>] _do_fork+0xe0/0x304
<4>[502718.915517] [<8002d8b0>] sys_fork+0x24/0x30
<4>[502718.919791] [<80019578>] syscall_common+0x34/0x58
<4>[502718.924567] Code: 00000000  8e020014  00e23821 <8ce20000> 10000009  cc400000  1040ffbd  00000000  8e060010 
<4>[502718.934403] 
<4>[502718.937417] ---[ end trace 06ca8e6fa072b8af ]---

===================================
Time: 1541399490.680810
<0>[502718.949217] Kernel panic - not syncing: Fatal exception
pparent76 commented 5 years ago

yet another crash:

<4>[27864.357390] Unhandled kernel unaligned access[#1]:
<4>[27864.362189] CPU: 0 PID: 8856 Comm: pidof Not tainted 4.14.78 #0
<4>[27864.368081] task: 8dc0e400 task.stack: 8e836000
<4>[27864.372586] $ 0   : 00000000 00000001 84d20db8 00000001
<4>[27864.377796] $ 4   : 80604dbf 00000000 00000000 00000000
<4>[27864.383003] $ 8   : 00000000 00000000 00000000 002fa969
<4>[27864.388211] $12   : 7fe8bbf8 00000000 00000000 77f332c0
<4>[27864.393419] $16   : 80604dc0 80604dbf 8fd9ea40 805a61b8
<4>[27864.398627] $20   : 80604dc0 8fd9ee58 8e837c5c 00000002
<4>[27864.403835] $24   : 00000fa3 77e96040                  
<4>[27864.409043] $28   : 8e836000 8e837c00 00000000 80051080
<4>[27864.414253] Hi    : 00000003
<4>[27864.417114] Lo    : 00000000
<4>[27864.420014] epc   : 8048f3a8 _raw_spin_lock+0xc/0x4c
<4>[27864.424970] ra    : 80051080 task_rq_lock+0x78/0x110
<4>[27864.429906] Status: 11008402      KERNEL EXL 
<4>[27864.433815] Cause : 40800010 (ExcCode 04)
<4>[27864.437799] BadVA : 80604dbf
<4>[27864.440662] PrId  : 0001992f (MIPS 1004Kc)
<4>[27864.444732] Modules linked in: xt_coova rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter em_cmp sch_multiq sch_teql em_nbyte sch_pie sch_gred sch_dsmark
<4>[27864.515922]  act_police cls_basic act_ipt sch_prio em_meta em_text sch_codel sch_sfq sch_fq sch_red act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug ext4 jbd2 mbcache
<4>[27864.586314]  usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[27864.593775] Process pidof (pid: 8856, threadinfo=8e836000, task=8dc0e400, tls=77f34d98)
<4>[27864.601734] Stack : 8fd9ea40 80051058 8e7ff000 80541584 8e837ccc 80541586 00000002 8fd9ea40
<4>[27864.610066]         8e837c98 8fd9ea40 805ad490 8fd9ea40 8da91ee0 00000000 00000000 80056998
<4>[27864.618396]         805a0000 00000f1e 8e837c64 80488170 805d0000 8da91ffc fffffd03 00000000
<4>[27864.626727]         8e837d60 8e837d68 8fd9ea40 805ad490 00000000 00000000 8da91ee0 00000001
<4>[27864.635057]         00000001 80056b38 000000e2 0000a206 811e2cc0 00000000 00000000 00000000
<4>[27864.643387]         ...
<4>[27864.645823] Call Trace:
<4>[27864.648259] [<8048f3a8>] _raw_spin_lock+0xc/0x4c
<4>[27864.652858] [<80051080>] task_rq_lock+0x78/0x110
<4>[27864.657457] [<80056998>] thread_group_cputime+0x110/0x1e0
<4>[27864.662830] [<80056b38>] thread_group_cputime_adjusted+0x28/0x60
<4>[27864.668836] [<80176538>] do_task_stat+0x330/0xaf4
<4>[27864.673518] [<801777fc>] proc_tgid_stat+0x18/0x24
<4>[27864.678200] [<80170874>] proc_single_show+0x5c/0xc4
<4>[27864.683065] [<8013cbec>] seq_read+0x270/0x570
<4>[27864.687414] [<801138b4>] __vfs_read+0x28/0x158
<4>[27864.691837] [<80113ab4>] vfs_read+0xd0/0x17c
<4>[27864.696088] [<80114034>] SyS_read+0x58/0xc4
<4>[27864.700267] [<80019578>] syscall_common+0x34/0x58
<4>[27864.704948] Code: 27bdfff8  0000000f  24030001 <c0850000> 14a00005  00000000  00600825  e0810000  1020fffa 
<4>[27864.714670] 
<4>[27864.717456] ---[ end trace 27617928f77f4b4e ]---
pparent76 commented 5 years ago

New crash:

<1>[183789.655235] CPU 2 Unable to handle kernel paging request at virtual address 07406000, epc == 8010c5c0, ra == 8010c4a4
<4>[183789.665917] Oops[#1]:
<4>[183789.668274] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.14.78 #0
<4>[183789.674338] task: 8fc444c0 task.stack: 8fc66000
<4>[183789.678929] $ 0   : 00000000 00000001 00000000 814b36f0
<4>[183789.684226] $ 4   : 805a61b8 00000001 00000001 07406000
<4>[183789.689522] $ 8   : 00184d39 00184d38 8f104830 8cc5c000
<4>[183789.694819] $12   : 805a0000 ffffffff 8e6c1930 00000000
<4>[183789.700117] $16   : 8fc02a00 01090220 8e410000 803415b0
<4>[183789.705413] $20   : 00000800 8e7aa05c 01080020 01080020
<4>[183789.710709] $24   : 0000000a 8039d4b4                  
<4>[183789.716006] $28   : 8fc66000 8fc11bc8 805a68f0 8010c4a4
<4>[183789.721306] Hi    : 00001135
<4>[183789.724255] Lo    : e9012000
<4>[183789.727246] epc   : 8010c5c0 __kmalloc_track_caller+0x1d4/0x228
<4>[183789.733227] ra    : 8010c4a4 __kmalloc_track_caller+0xb8/0x228
<4>[183789.739116] Status: 11007c03     KERNEL EXL IE 
<4>[183789.743373] Cause : 40800008 (ExcCode 02)
<4>[183789.747445] BadVA : 07406000
<4>[183789.750395] PrId  : 0001992f (MIPS 1004Kc)
<4>[183789.754552] Modules linked in: xt_coova rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter em_cmp sch_multiq sch_teql em_nbyte sch_pie sch_gred sch_dsmark
<4>[183789.825837]  act_police cls_basic act_ipt sch_prio em_meta em_text sch_codel sch_sfq sch_fq sch_red act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug ext4 jbd2 mbcache
<4>[183789.896336]  usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[183789.903915] Process swapper/2 (pid: 0, threadinfo=8fc66000, task=8fc444c0, tls=00000000)
<4>[183789.912054] Stack : 00000000 00000001 80606694 8e9765b0 8cc69f00 00000000 01080020 8033f6d8
<4>[183789.920492]         8e60c000 8e9765f0 8e976648 00000014 8cc69f00 8fec4280 00000000 00000740
<4>[183789.928930]         000005dc 803415b0 8f177918 8e60cd48 8cc69f00 ac07f557 8cc69f00 8cc69f00
<4>[183789.937369]         8fec4280 805c77e0 805c0000 000005dc 8e7aa05c 01080020 00000000 8039d7cc
<4>[183789.945805]         00000005 80393900 8e382864 8fc11ca4 8e997380 80065c10 805a0000 80339980
<4>[183789.954242]         ...
<4>[183789.956773] Call Trace:
<4>[183789.959329] [<8010c5c0>] __kmalloc_track_caller+0x1d4/0x228
<4>[183789.964992] [<8033f6d8>] __kmalloc_reserve.isra.7+0x40/0xa4
<4>[183789.970640] [<803415b0>] pskb_expand_head+0x88/0x318
<4>[183789.975679] [<8039d7cc>] ip_forward+0x318/0x49c
<4>[183789.980285] [<8039c1e0>] ip_rcv+0x260/0x2e0
<4>[183789.984567] [<80352200>] __netif_receive_skb_core+0xa94/0xc5c
<4>[183789.990401] [<80358168>] netif_receive_skb_internal+0xd8/0xf0
<4>[183789.996224] [<8045e530>] br_pass_frame_up+0xe8/0x154
<4>[183790.001261] [<8045eb28>] br_handle_frame_finish+0x534/0x578
<4>[183790.006906] [<8045ee9c>] br_handle_frame+0x330/0x3d4
<4>[183790.011945] [<80351ef4>] __netif_receive_skb_core+0x788/0xc5c
<4>[183790.017761] [<80354d6c>] process_backlog+0x98/0x160
<4>[183790.022711] [<803585d4>] net_rx_action+0x150/0x30c
<4>[183790.027578] [<8048f920>] __do_softirq+0x128/0x2ec
<4>[183790.032369] [<80032b34>] irq_exit+0xac/0xc8
<4>[183790.036632] [<802511ac>] plat_irq_dispatch+0xfc/0x138
<4>[183790.041765] [<8000b5e8>] except_vec_vi_end+0xb8/0xc4
<4>[183790.046809] [<8000cfb0>] r4k_wait_irqoff+0x1c/0x24
<4>[183790.051700] [<8006687c>] do_idle+0xe4/0x168
<4>[183790.055958] [<80066af8>] cpu_startup_entry+0x24/0x2c
<4>[183790.061009] [<800180b4>] not_nmi+0x8c/0x1d8
<4>[183790.065268] Code: 00000000  8e020014  00e23821 <8ce20000> 10000009  cc400000  1040ffbd  00000000  8e060010 
<4>[183790.075100] 
<4>[183790.076956] ---[ end trace 45577ca1cf2e760c ]---

===================================
Time: 1541941188.747377
camelcamro commented 5 years ago

also having crashes 1-2 each day... but can't connect to it, neither there is no crash log after reboot. (trunk)

pparent76 commented 5 years ago

@camelcamro there should be crashlogs I'm running trunk snapshots.

araujorm commented 5 years ago

Hello.

I have 3 routers, all with latest mt76 master. I can confirm that they are all rebooting (they always have been, but sometimes are worse than others) and that sometimes no crashlog is available. Other times, they are a bit useless since the trace only contains calls that seem to me to be too deep inside the kernel so I usually refrain from posting them in order not to clutter too much the issue threads.

Last Friday I posted one that seemed to be useful to me in https://github.com/openwrt/mt76/issues/198 although I'm not sure if it was related to that thread. Coincidently or not, a bit later @nbd168 made a new commit that seemed to me to be related, so I git pulled and built a fresh image and reflashed my 3 routers

Unfortunately, the crashes and hangups still happened during the weekend with no (useful) crashlog, until I rechecked a while ago and one of them produced the following, which seems interesting. @nbd168, can you please see if it leads to something? Thanks in advance.

Version is OpenWrt SNAPSHOT, r8444-7dd42aa592, and the important crashlog excerpt is:

<1>[53793.996849] CPU 0 Unable to handle kernel paging request at virtual address 00000014, epc == 80222aec, ra == 80222b28
<4>[53794.017904] Oops[#1]:
<4>[53794.022400] CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.79 #0
<4>[53794.033942] task: 8039f310 task.stack: 8039a000
<4>[53794.042897] $ 0   : 00000000 00000001 8336ff40 8336ffe8
<4>[53794.053241] $ 4   : 00000000 00000001 00000024 00000001
<4>[53794.063585] $ 8   : 81066b54 0000006c 0000003c 00000004
<4>[53794.073930] $12   : ffffff80 00000001 ffffff80 00000100
<4>[53794.084274] $16   : 82c85e40 00000015 00000000 00000001
<4>[53794.094618] $20   : 0000db3d 00000001 00000000 00000000
<4>[53794.104961] $24   : 00000200 ffffffff                  
<4>[53794.115306] $28   : 8039a000 83809d98 00000000 80222b28
<4>[53794.125653] Hi    : 00000000
<4>[53794.131341] Lo    : 0000006c
<4>[53794.137052] epc   : 80222aec __pskb_pull_tail+0x264/0x3c0
<4>[53794.147737] ra    : 80222b28 __pskb_pull_tail+0x2a0/0x3c0
<4>[53794.158413] Status: 1100f403  KERNEL EXL IE 
<4>[53794.166697] Cause : 00800008 (ExcCode 02)
<4>[53794.174620] BadVA : 00000014
<4>[53794.180309] PrId  : 00019655 (MIPS 24KEc)
<4>[53794.188232] Modules linked in: mt76x2e mt76x2_common mt76x02_lib mt7603e mt76 mac80211 cfg80211 compat leds_gpio gpio_button_hotplug
<4>[53794.211859] Process swapper (pid: 0, threadinfo=8039a000, task=8039f310, tls=00000000)
<4>[53794.227523] Stack : 00000000 803a0000 80420000 8008d470 82c85e40 00000080 83aecbc0 00000000
<4>[53794.244077]         00000000 00000006 83ac6df0 83b25ce4 0ee6b280 00000000 80420000 8005ecc8
<4>[53794.260625]         83215b00 83057bd0 83aecbc0 83809e88 803a53e8 8039f310 00000000 00000003
<4>[53794.277175]         000000d0 83aed918 82c85e40 83aecbc0 00000000 00000000 00000000 00000000
<4>[53794.293724]         00000000 00000000 00000000 00000000 82c85e40 83aed9f8 83809e9c 83aed4a0
<4>[53794.310272]         ...
<4>[53794.315105] Call Trace:
<4>[53794.319941] [<80222aec>] __pskb_pull_tail+0x264/0x3c0
<4>[53794.330096] [<83b25ce4>] ieee80211_rx_napi+0x4a0/0x960 [mac80211]
<4>[53794.342220] [<83072430>] mt76_rx_complete+0x134/0x174 [mt76]
<4>[53794.353430] [<83072668>] mt76_rx_poll_complete+0x1f8/0x334 [mt76]
<4>[53794.365500] [<8307106c>] mt76_dma_tx_queue_skb+0xc88/0x10e0 [mt76]
<4>[53794.377733] Code: 1480001a  26240005  8c64002c <8c820014> 30430001  10600002  00000000  2444ffff  24820010 
<4>[53794.397051] 
<4>[53794.400045] ---[ end trace bef1f9d576dde985 ]---

Please note that my images have a minimal custom config, with less stuff than the vanilla one, in particular they have no netfilter/iptables or ipv6, and I always compile them myself. Also, they have a custom patch I'm working on to try to better support those particular devices (a new DTS for Pixlink AC05 1200, which are more or less clones of Wavlink WL-WN575A3B but the leds and gpio pins aren't exactly the same, should anyone want them then feel free to ask), and a small change so that I can build the ethernet driver without netfilter/iptables without errors. Those patches don't touch anything related to mt76 drivers or anything else that could be related for that matter, so I'm positive they don't interfere, but still here's the disclaimer.

nbd168 commented 5 years ago

Please try latest OpenWrt master

pparent76 commented 5 years ago

Ok my router is up to date with 4.14.80+2018-11-13-88f68838-2 . As usual I keep you posted in few days about the result.

araujorm commented 5 years ago

Hello.

Since yesterday evening, OpenWrt SNAPSHOT r8455-042d68a195 (latest mt76 master), I've been having even more crashes :(

But now, most of the time, the routers simply hang and I have no option but to restart them manually, losing any crashlog.

Still, I was able to get one, hope it helps:

<1>[ 3012.758128] CPU 0 Unable to handle kernel paging request at virtual address 07406000, epc == 800bd3b8, ra == 800bd2d0
<4>[ 3012.779182] Oops[#1]:
<4>[ 3012.783678] CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.80 #0
<4>[ 3012.795220] task: 8039f310 task.stack: 8039a000
<4>[ 3012.804178] $ 0   : 00000000 00000001 07406000 00000000
<4>[ 3012.814523] $ 4   : 00000001 0000a967 0000a966 01080020
<4>[ 3012.824868] $ 8   : 00000002 00000001 00000010 8820e000
<4>[ 3012.835214] $12   : 00000006 035b6e50 00000000 00000000
<4>[ 3012.845559] $16   : 83802a00 83168000 01090220 80220dfc
<4>[ 3012.855904] $20   : 00000720 00000000 00000006 01080020
<4>[ 3012.866250] $24   : 00000010 8018b650                  
<4>[ 3012.876595] $28   : 8039a000 83809ac0 83ac8bc0 800bd2d0
<4>[ 3012.886942] Hi    : 000007cf
<4>[ 3012.892631] Lo    : 00000001
<4>[ 3012.898340] epc   : 800bd3b8 __kmalloc_track_caller+0x19c/0x1b0
<4>[ 3012.910053] ra    : 800bd2d0 __kmalloc_track_caller+0xb4/0x1b0
<4>[ 3012.921590] Status: 1100f403      KERNEL EXL IE 
<4>[ 3012.929873] Cause : 00800008 (ExcCode 02)
<4>[ 3012.937796] BadVA : 07406000
<4>[ 3012.943486] PrId  : 00019655 (MIPS 24KEc)
<4>[ 3012.951408] Modules linked in: mt76x2e mt76x2_common mt76x02_lib mt7603e mt76 mac80211 cfg80211 compat leds_gpio gpio_button_hotplug
<4>[ 3012.975035] Process swapper (pid: 0, threadinfo=8039a000, task=8039f310, tls=00000000)
<4>[ 3012.990698] Stack : 00000018 8302ecee 838da480 00000000 838dae40 01080020 00000000 8021f090
<4>[ 3013.007248]         83bccbc0 83053440 0000004e 00000000 838dae40 00000018 8302ecee 00000660
<4>[ 3013.023795]         00000000 80220dfc 00000000 00000000 00000000 838da480 83809b18 83ac8bc0
<4>[ 3013.040342]         00000018 8302ecee 838dae40 00000000 00000000 00000006 00000000 83b28c54
<4>[ 3013.056889]         838da480 83053440 fffffffe 00000000 83a7e440 83b295b8 00000000 00000004
<4>[ 3013.073436]         ...
<4>[ 3013.078266] Call Trace:
<4>[ 3013.083100] [<800bd3b8>] __kmalloc_track_caller+0x19c/0x1b0
<4>[ 3013.094138] [<8021f090>] __kmalloc_reserve.isra.8+0x40/0xa4
<4>[ 3013.105170] [<80220dfc>] pskb_expand_head+0x88/0x2c4
<4>[ 3013.115148] [<83b28c54>] ieee80211_nullfunc_get+0xb70/0x16b4 [mac80211]
<4>[ 3013.128286] Code: 00000000  8e030014  00431021 <8c420000> 1000ffd4  cc400000  1000fff2  00002025  3c028042 
<4>[ 3013.147603] 
<4>[ 3013.150582] ---[ end trace 37570532a6293b35 ]---

Meanwhile I'll have to revert, since they were mostly unusable during the day.

Best regards.

nbd168 commented 5 years ago

Please try the latest version

pparent76 commented 5 years ago

I can confirm hangs with 4.14.80+2018-11-13-88f68838-2 . Router does not respond anymore to ssh. I will try later version on monday.

nbd168 commented 5 years ago

I pushed some more fixes, please try the latest version.

pataha commented 5 years ago

new crash with r8474-0191a06 xiaomi r3g, wifi does not show up anymore until restart the router

[   18.987694] br-lan: port 1(eth0.1) entered blocking state
[   18.993129] br-lan: port 1(eth0.1) entered disabled state
[   18.999488] device eth0.1 entered promiscuous mode
[   19.008240] br-lan: port 1(eth0.1) entered blocking state
[   19.013690] br-lan: port 1(eth0.1) entered forwarding state
[   19.019867] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[   19.776761] random: crng init done
[   20.037153] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[   21.300019] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[   21.309976] br-lan: port 2(wlan0) entered blocking state
[   21.315283] br-lan: port 2(wlan0) entered disabled state
[   21.321397] device wlan0 entered promiscuous mode
[   21.720904] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[   21.727836] br-lan: port 2(wlan0) entered blocking state
[   21.733155] br-lan: port 2(wlan0) entered forwarding state
[  482.502232] ------------[ cut here ]------------
[  482.506910] WARNING: CPU: 1 PID: 1736 at /builder/ramips_mt7621/build/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7621/mt76-2018-11-21-da666a74/mt7603/mac.c:196 mt7603_filter_tx+0x100/0x164 [mt7603e]
[  482.525016] Modules linked in: pppoe ppp_async pppox ppp_generic nf_conntrack_ipv6 mt76x2e mt76x2_common mt76x02_lib mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables crc_ccitt compat ledtrig_usbport ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd gpio_button_hotplug usbcore nls_base usb_common
[  482.595898] CPU: 1 PID: 1736 Comm: hostapd Not tainted 4.14.81 #0
[  482.601964] Stack : 00000000 00000001 803907b0 8006f714 805d0000 8056dd28 00000000 00000000
[  482.610302]         80538698 8ea63a54 8fdd963c 805a8947 80533788 00000001 8ea639f8 1cc282b1
[  482.618636]         00000000 00000000 80610000 00005d78 00000000 00000127 00000008 00000000
[  482.626973]         00000000 805b0000 000917ba 00000000 80000000 00000000 805d0000 8f1743a0
[  482.635307]         00000009 000000c4 8f170000 00000001 00000000 802a2cc8 00000004 80610004
[  482.643637]         ...
[  482.646073] Call Trace:
[  482.648535] [<800106c0>] show_stack+0x58/0x100
[  482.652988] [<8047269c>] dump_stack+0x9c/0xe0
[  482.657336] [<8002e368>] __warn+0xe0/0x114
[  482.661414] [<8002e42c>] warn_slowpath_null+0x1c/0x30
[  482.666455] [<8f1743a0>] mt7603_filter_tx+0x100/0x164 [mt7603e]
[  482.672351] [<8f171578>] mt7603_sta_remove+0x84/0xb0 [mt7603e]
[  482.678174] [<8f152090>] mt76_sta_state+0x110/0x300 [mt76]
[  482.683762] [<8f287c60>] sta_set_sinfo+0xbf8/0xcd0 [mac80211]
[  482.689613] ---[ end trace d4a1a16b771dfadf ]---
BertLammers commented 5 years ago

I encounter the same crash (see below). 5Ghz is still working fine, but the 2.4Ghz network is gone and a restart of the router (Xiaomi R3G) is needed to get it working again. The 2.4Ghz network seemed to perform stable until the crash.

Using OpenWrt SNAPSHOT, r8474+20-0191a06


[   29.254643] br-lan: port 2(wlan0) entered blocking state
[   29.260082] br-lan: port 2(wlan0) entered forwarding state
[   91.333265] IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready
[   91.339950] br-lan: port 3(wlan1) entered blocking state
[   91.345310] br-lan: port 3(wlan1) entered forwarding state
[ 4533.695583] mtk_soc_eth 1e100000.ethernet eth0: port 2 link up
[ 4533.701678] mtk_soc_eth 1e100000.ethernet eth0: port 2 link down
[ 4536.233265] mtk_soc_eth 1e100000.ethernet eth0: port 2 link up
[ 4536.239359] mtk_soc_eth 1e100000.ethernet eth0: port 2 link down
[ 4537.913235] mtk_soc_eth 1e100000.ethernet eth0: port 2 link up
[10002.151247] ------------[ cut here ]------------
[10002.155924] WARNING: CPU: 1 PID: 3063 at /home/bert/xiaomi-mir3g/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7621/mt76-2018-11-21-da666a74/mt7603/mac.c:196 mt7603_filter_tx+0x100/0x164 [mt7603e]
[10002.173603] Modules linked in: pppoe ppp_async pppox ppp_generic nf_conntrack_ipv6 mt76x2e mt76x2_common mt76x02_lib mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY wireguard slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat fuse sch_cake nf_conntrack act_skbedit act_mirred em_u32 cls_u32
[10002.244505]  cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables ifb ip6_udp_tunnel udp_tunnel usb_storage leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd sd_mod scsi_mod gpio_button_hotplug usbcore nls_base usb_common
[10002.276717] CPU: 1 PID: 3063 Comm: hostapd Not tainted 4.14.81 #0
[10002.282782] Stack : 00000000 00000001 80377750 8006f564 80590000 8053906c 00000000 00000000
[10002.291117]         80505148 8d243a54 8fd8e77c 805728c7 80500238 00000001 8d2439f8 1cc282b1
[10002.299454]         00000000 00000000 805d0000 00006aa0 00000000 00000153 00000008 00000000
[10002.307789]         00000000 80570000 000438ed 00000000 80000000 00000000 80590000 8e3843a0
[10002.316124]         00000009 000000c4 8e380000 00000001 00000000 8028fd90 00000004 805d0004
[10002.324460]         ...
[10002.326900] Call Trace:
[10002.329364] [<800106c0>] show_stack+0x58/0x100
[10002.333802] [<804447ac>] dump_stack+0x9c/0xe0
[10002.338151] [<8002e368>] __warn+0xe0/0x114
[10002.342232] [<8002e42c>] warn_slowpath_null+0x1c/0x30
[10002.347280] [<8e3843a0>] mt7603_filter_tx+0x100/0x164 [mt7603e]
[10002.353180] [<8e381578>] mt7603_sta_remove+0x84/0xb0 [mt7603e]
[10002.359016] [<8e2fa090>] mt76_sta_state+0x110/0x300 [mt76]
[10002.364610] [<8ea07c60>] sta_set_sinfo+0xbf8/0xcd0 [mac80211]
[10002.370457] ---[ end trace c2d577232df83e4e ]---
[11239.733409] mtk_soc_eth 1e100000.ethernet eth0: port 2 link down
camelcamro commented 5 years ago

maybe mine is also related to the same issue

<1>[162146.511610] CPU 1 Unable to handle kernel paging request at virtual address 00000628, epc == 8010c560, ra == 8010c444 <4>[162146.522422] Oops[#1]: <4>[162146.524889] CPU: 1 PID: 18638 Comm: tc Not tainted 4.14.81 #0 <4>[162146.530762] task: 8fe18000 task.stack: 8e584000 <4>[162146.535375] $ 0 : 00000000 00000001 00000000 814a5700 <4>[162146.540686] $ 4 : 805a61b8 00000001 00000001 00000628 <4>[162146.545994] $ 8 : 03ac679b 03ac679a 00000000 80000000 <4>[162146.551302] $12 : 8e584000 00000000 00000000 77fd32c0 <4>[162146.556613] $16 : 8fc02900 014102c0 8e506000 8033f818 <4>[162146.561923] $20 : 00001000 8da44300 00000b98 00000000 <4>[162146.567230] $24 : 00000000 8038e438 <4>[162146.572538] $28 : 8e584000 8e585cb0 00000000 8010c444 <4>[162146.577849] Hi : 08418c00 <4>[162146.580803] Lo : 07395a80 <4>[162146.583798] epc : 8010c560 __kmalloc_track_caller+0x1d4/0x228 <4>[162146.589789] ra : 8010c444 __kmalloc_track_caller+0xb8/0x228 <4>[162146.595681] Status: 11007c03 KERNEL EXL IE <4>[162146.599948] Cause : 40800008 (ExcCode 02) <4>[162146.604022] BadVA : 00000628 <4>[162146.606977] PrId : 0001992f (MIPS 1004Kc) <4>[162146.611139] Modules linked in: pppoe ppp_async option usb_wwan pptp pppox ppp_mppe ppp_generic nf_nat_pptp nf_conntrack_pptp nf_conntrack_ipv6 mt76x2e mt76x2_common mt76x02_lib mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_NETMAP xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY usbserial usb_storage ts_fsm ts_bm slhc nf_reject_ipv4 nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_rtsp nf_nat_redirect nf_nat_proto_gre nf_nat_masquerade_ipv4 nf_nat_irc nf_conntrack_ipv4 nf_nat_ipv4 nf_nat_h323 nf_nat_amanda nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table <4>[162146.682805] nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_rtsp nf_conntrack_rtcache nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_irc nf_conntrack_h323 nf_conntrack_broadcast ts_kmp nf_conntrack_amanda macvlan libcrc32c iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter fuse sch_cake nf_conntrack act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip <4>[162146.754220] ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables nfsv4 nfsv3 nfs ip_gre gre ifb ip_tunnel tun vfat fat lockd sunrpc grace ext4 jbd2 mbcache dns_resolver dm_crypt dm_mirror dm_region_hash dm_log dm_mod dax nls_utf8 nls_iso8859_1 nls_cp437 sha1_generic ecb crc32c_generic mmc_block mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug usbcore nls_base usb_common <4>[162146.799627] Process tc (pid: 18638, threadinfo=8e584000, task=8fe18000, tls=77fd4d98) <4>[162146.807507] Stack : 00000000 80338f38 00000000 00000000 8e815000 8e585d0f 014000c0 8033f718 <4>[162146.815946] 00000000 8e585d50 8e585d54 fffffff5 8e815000 00000f5f 014000c0 8fc20600 <4>[162146.824385] 00000000 8033f818 00000000 00000b98 00000000 00000000 8e585ee8 8da44300 <4>[162146.832821] 8e815b40 8e3e0800 8e815b40 00000f40 00000f24 8e585ee8 8da44300 8038e1ec <4>[162146.841257] 00000000 803477f8 8e3e0800 8e815b40 8e3e0800 8034657c 8e585ee8 8e3e0800 <4>[162146.849695] ... <4>[162146.852225] Call Trace: <4>[162146.854761] [<8010c560>] __kmalloc_track_caller+0x1d4/0x228 <4>[162146.860427] [<8033f718>] __kmalloc_reserve.isra.7+0x40/0xa4 <4>[162146.866075] [<8033f818>] __alloc_skb+0x9c/0x180 <4>[162146.870692] [<8038e1ec>] netlink_dump+0x84/0x2d0 <4>[162146.875387] [<8038e608>] netlink_recvmsg+0x1d0/0x344 <4>[162146.880449] [<80335818>] ___sys_recvmsg+0xc0/0x178 <4>[162146.885320] [<803368dc>] __sys_recvmsg+0x50/0x80 <4>[162146.890029] [<80019578>] syscall_common+0x34/0x58 <4>[162146.894807] Code: 00000000 8e020014 00e23821 <8ce20000> 10000009 cc400000 1040ffbd 00000000 8e060010 <4>[162146.904644] <4>[162146.907236] ---[ end trace 294fcbb12d1cc3ec ]---
BertLammers commented 5 years ago

Running OpenWrt SNAPSHOT, r8598+21-8ba6e95 for over 5 days now. No crashes anymore, wifi seems to perform well both on 2.4 and 5ghz.

Thanks for the hard work!

pparent76 commented 5 years ago

New crash for me:


<1>[60139.037498] raw: 40040068 8dcd4a51 00000094 fffffffe 00000001 81436974 8143d754 00000000
<1>[60139.045600] page dumped because: bad pte
<1>[60139.049508] addr:77fb2000 vm_flags:00100877 anon_vma:8dcd4a50 mapping:8f86244c index:94
<1>[60139.057593] file:libc.so fault:filemap_fault mmap:generic_file_readonly_mmap readpage:squashfs_readpage
<4>[60139.067030] CPU: 1 PID: 22203 Comm: sh Not tainted 4.14.87 #0
<4>[60139.072753] Stack : 00000000 81433620 21831b8f 8006f764 805d0000 80572f1c 00000000 00000000
<4>[60139.081088]         8053d73c 8eb7bb9c 8f1bce7c 805ae947 80538808 00000001 8eb7bb40 ac07f54e
<4>[60139.089427]         00000000 00000000 80610000 00007670 00000000 0000217b 00000007 00000000
<4>[60139.097765]         00000000 805b0000 000105d6 00000000 80000000 00000000 805d0000 80640000
<4>[60139.106104]         805b0000 77fb2000 8d93077c 81433620 00000000 802a3388 00000004 80610004
<4>[60139.114441]         ...
<4>[60139.116879] Call Trace:
<4>[60139.119349] [<800106c0>] show_stack+0x58/0x100
<4>[60139.123784] [<804772ac>] dump_stack+0x9c/0xe0
<4>[60139.128141] [<800ec584>] print_bad_pte+0x1e8/0x220
<4>[60139.132915] [<800eea58>] unmap_page_range+0x454/0x810
<4>[60139.137947] [<800ef150>] unmap_vmas+0x34/0x78
<4>[60139.142307] [<800f5a60>] exit_mmap+0xfc/0x190
<4>[60139.146654] [<8002bb2c>] mmput+0x44/0x108
<4>[60139.150650] [<8011b3d0>] flush_old_exec+0x714/0x808
<4>[60139.155513] [<80168ab4>] load_elf_binary+0x448/0x1354
<4>[60139.160552] [<8011b56c>] search_binary_handler.part.9+0xa8/0x234
<4>[60139.166545] [<8011bed4>] do_execveat_common+0x4d8/0x6a4
<4>[60139.171759] [<8011c2cc>] SyS_execve+0x34/0x48
<4>[60139.176118] [<80019578>] syscall_common+0x34/0x58
<4>[60139.180983] Disabling lock debugging due to kernel taint
<1>[60139.186578] BUG: Bad rss-counter state mm:8e647880 idx:0 val:1
<1>[60139.192508] BUG: Bad rss-counter state mm:8e647880 idx:1 val:-1
<1>[60139.212775] BUG: Bad page map in process sh  pte:25891b0b pmd:8ed18000
<1>[60139.219370] addr:7fffe000 vm_flags:00118177 anon_vma:8e076600 mapping:  (null) index:7fffe
<1>[60139.227688] file:  (null) fault:  (null) mmap:  (null) readpage:  (null)
<1>[60139.227769] BUG: Bad rss-counter state mm:8e002380 idx:0 val:1
<1>[60139.227777] BUG: Bad rss-counter state mm:8e002380 idx:1 val:-1
<4>[60139.246245] CPU: 3 PID: 22206 Comm: sh Tainted: G    B           4.14.87 #0
<4>[60139.253185] Stack : 00000000 00000000 25891b0b 8006f764 805d0000 80572f1c 00000000 00000000
<4>[60139.261524]         8053d73c 8e4efca4 8e22483c 805ae947 80538808 00000001 8e4efc48 ac07f54e
<4>[60139.269862]         00000000 00000000 80610000 00007e38 00000000 00002198 00000007 00000000
<4>[60139.278197]         00000000 805b0000 0003c1e5 00000000 80000000 00000000 805d0000 7fffe000
<4>[60139.286532]         00000ff8 7fffe000 8ea147fc 00000000 00000002 802a3388 0000000c 8061000c
<4>[60139.294869]         ...
<4>[60139.297308] Call Trace:
<4>[60139.299774] [<800106c0>] show_stack+0x58/0x100
<4>[60139.304209] [<804772ac>] dump_stack+0x9c/0xe0
<4>[60139.308556] [<800ec584>] print_bad_pte+0x1e8/0x220
<4>[60139.313327] [<800edf60>] _vm_normal_page+0xac/0xdc
<4>[60139.318104] [<800eaad4>] follow_pmd_mask.isra.4+0x1a8/0x470
<4>[60139.323651] [<800eaf2c>] __get_user_pages.part.7+0x190/0x36c
<4>[60139.329289] [<800eb654>] get_user_pages_remote+0xcc/0x214
<4>[60139.334672] [<80119d10>] copy_strings+0x150/0x3f8
<4>[60139.339358] [<8011be70>] do_execveat_common+0x474/0x6a4
<4>[60139.344563] [<8011c2cc>] SyS_execve+0x34/0x48
<4>[60139.348915] [<80019578>] syscall_common+0x34/0x58
<1>[60139.353788] BUG: Bad page map in process sh  pte:25891b0b pmd:8ed18000
<1>[60139.360338] addr:7fffe000 vm_flags:00118177 anon_vma:8e076600 mapping:  (null) index:7fffe
<1>[60139.368641] file:  (null) fault:  (null) mmap:  (null) readpage:  (null)
<4>[60139.375382] CPU: 3 PID: 22206 Comm: sh Tainted: G    B           4.14.87 #0
<4>[60139.382312] Stack : 00000000 00000000 25891b0b 8006f764 805d0000 80572f1c 00000000 00000000
<4>[60139.390647]         8053d73c 8e4efcbc 8e22483c 805ae947 80538808 00000001 8e4efc60 ac07f54e
<4>[60139.398987]         00000000 00000000 80610000 00008430 00000000 000021ae 00000007 00000000
<4>[60139.407324]         00000000 805b0000 0005ba56 00000000 80000000 00000000 805d0000 25891b0b
<4>[60139.415659]         805abce0 7fffe000 8ea147fc 00000000 00000002 802a3388 0000000c 8061000c
<4>[60139.423993]         ...
<4>[60139.426430] Call Trace:
<4>[60139.428887] [<800106c0>] show_stack+0x58/0x100
<4>[60139.433316] [<804772ac>] dump_stack+0x9c/0xe0
<4>[60139.437667] [<800ec584>] print_bad_pte+0x1e8/0x220
<4>[60139.442440] [<800edf60>] _vm_normal_page+0xac/0xdc
<4>[60139.447213] [<800ee7b4>] unmap_page_range+0x1b0/0x810
<4>[60139.452241] [<800ef150>] unmap_vmas+0x34/0x78
<4>[60139.456597] [<800f5a60>] exit_mmap+0xfc/0x190
<4>[60139.460938] [<8002bb2c>] mmput+0x44/0x108
<4>[60139.464944] [<8011bf90>] do_execveat_common+0x594/0x6a4
<4>[60139.470149] [<8011c2cc>] SyS_execve+0x34/0x48
<4>[60139.474504] [<80019578>] syscall_common+0x34/0x58
<1>[60139.479493] BUG: Bad rss-counter state mm:8e0036c0 idx:1 val:1
<1>[60139.485871] BUG: Bad page map in process sh  pte:21831b8f pmd:8ff35000
<0>[60139.492381] page:81433620 count:1 mapcount:-1 mapping:8e076241 index:0xbe0
<0>[60139.499336] flags: 0x40040068(uptodate|lru|active|swapbacked)
<1>[60139.505156] raw: 40040068 8e076241 00000be0 fffffffe 00000001 81441f54 81443114 00000000
<1>[60139.513211] page dumped because: bad pte
<1>[60139.517203] addr:00be0000 vm_flags:00100077 anon_vma:8e076240 mapping:  (null) index:be0
<1>[60139.525304] file:  (null) fault:  (null) mmap:  (null) readpage:  (null)
<4>[60139.531984] CPU: 3 PID: 22206 Comm: sh Tainted: G    B           4.14.87 #0
<4>[60139.538912] Stack : 00000000 81433620 21831b8f 8006f764 805d0000 80572f1c 00000000 00000000
<4>[60139.547251]         8053d73c 8e4efcbc 8e22483c 805ae947 80538808 00000001 8e4efc60 ac07f54e
<4>[60139.555585]         00000000 00000000 80610000 00008b80 00000000 000021c9 00000007 00000000
<4>[60139.563917]         00000000 805b0000 00081e10 00000000 80000000 00000000 805d0000 81433620
<4>[60139.572248]         805abce0 00be0000 8f3fc008 81433620 00000002 802a3388 0000000c 8061000c
<4>[60139.580585]         ...
<4>[60139.583028] Call Trace:
<4>[60139.585495] [<800106c0>] show_stack+0x58/0x100
<4>[60139.589927] [<804772ac>] dump_stack+0x9c/0xe0
<4>[60139.594278] [<800ec584>] print_bad_pte+0x1e8/0x220
<4>[60139.599051] [<800eea58>] unmap_page_range+0x454/0x810
<4>[60139.604083] [<800ef150>] unmap_vmas+0x34/0x78
<4>[60139.608429] [<800f5a60>] exit_mmap+0xfc/0x190
<4>[60139.612767] [<8002bb2c>] mmput+0x44/0x108
<4>[60139.616769] [<80030928>] do_exit+0x380/0x9a0
<4>[60139.621021] [<80031df0>] SyS_exit_group+0x0/0x14
<1>[60139.630069] BUG: Bad rss-counter state mm:8e7fe8c0 idx:0 val:1
<1>[60139.635978] BUG: Bad rss-counter state mm:8e7fe8c0 idx:1 val:-1
<1>[60139.646949] BUG: Bad page map in process uci  pte:21831a8d pmd:8ef6b000
<0>[60139.653566] page:81433620 count:1 mapcount:-1 mapping:8e70cba1 index:0x94
<0>[60139.660422] flags: 0x40040068(uptodate|lru|active|swapbacked)
<1>[60139.666208] raw: 40040068 8e70cba1 00000094 fffffffe 00000001 81436934 81435054 00000000
<1>[60139.674375] page dumped because: bad pte
<1>[60139.678298] addr:7fd6e000 vm_flags:00100173 anon_vma:8e70c1e0 mapping:  (null) index:7ffff
<1>[60139.686595] file:  (null) fault:  (null) mmap:  (null) readpage:  (null)
<4>[60139.693293] CPU: 1 PID: 22207 Comm: uci Tainted: G    B           4.14.87 #0
<4>[60139.700311] Stack : 00000000 81433620 21831a8d 8006f764 805d0000 80572f1c 00000000 00000000
<4>[60139.708649]         8053d73c 8eb7bcbc 8fe822bc 805ae947 80538808 00000001 8eb7bc60 ac07f54e
<4>[60139.716990]         00000000 00000000 80610000 00009298 00000000 000021e3 00000007 00000000
<4>[60139.725332]         00000000 805b0000 000a942d 00000000 80000000 00000000 805d0000 81433620
<4>[60139.733673]         805abce0 7fd6e000 8e36e7fc 81433620 00000003 802a3388 00000004 80610004
<4>[60139.742011]         ...
<4>[60139.744450] Call Trace:
<4>[60139.746916] [<800106c0>] show_stack+0x58/0x100
<4>[60139.751347] [<804772ac>] dump_stack+0x9c/0xe0
<4>[60139.755702] [<800ec584>] print_bad_pte+0x1e8/0x220
<4>[60139.760486] [<800eea58>] unmap_page_range+0x454/0x810
<4>[60139.765520] [<800ef150>] unmap_vmas+0x34/0x78
<4>[60139.769868] [<800f5a60>] exit_mmap+0xfc/0x190
<4>[60139.774216] [<8002bb2c>] mmput+0x44/0x108
<4>[60139.778214] [<80030928>] do_exit+0x380/0x9a0
<4>[60139.782474] [<80031df0>] SyS_exit_group+0x0/0x14
<1>[60139.789333] CPU 2 Unable to handle kernel paging request at virtual address 00000104, epc == 800cf880, ra == 800cf76c
<4>[60139.799923] Oops[#1]:
<4>[60139.802190] CPU: 2 PID: 22208 Comm: sh Tainted: G    B           4.14.87 #0
<4>[60139.809123] task: 8dcb3e80 task.stack: 8da34000
<4>[60139.813629] $ 0   : 00000000 00000001 00000200 00000100
<4>[60139.818842] $ 4   : 805d4340 00000001 00000012 805ac1b8
<4>[60139.824056] $ 8   : 8060f275 00000000 00000000 00000000
<4>[60139.829266] $12   : 7fb26240 00000000 00000000 77fba2c0
<4>[60139.834478] $16   : 81467d00 00000001 805d4340 00000008
<4>[60139.839689] $20   : 00000002 00000001 805d4340 80640000
<4>[60139.844899] $24   : 00458ef8 77f36f98                  
<4>[60139.850111] $28   : 8da34000 8da35d28 00000002 800cf76c
<4>[60139.855325] Hi    : 00000017
<4>[60139.858189] Lo    : 00000046
<4>[60139.861086] epc   : 800cf880 __page_cache_release+0x190/0x22c
<4>[60139.866810] ra    : 800cf76c __page_cache_release+0x7c/0x22c
<4>[60139.872437] Status: 11007c02      KERNEL EXL 
<4>[60139.876351] Cause : 4080000c (ExcCode 03)
<4>[60139.880337] BadVA : 00000104
<4>[60139.883199] PrId  : 0001992f (MIPS 1004Kc)
<4>[60139.887271] Modules linked in: xt_coova rtl_usb pppoe ppp_async rtlwifi pppox ppp_generic nf_conntrack_ipv6 mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat br_netfilter em_cmp sch_teql em_nbyte sch_dsmark sch_pie sch_codel
<4>[60139.957609]  sch_gred cls_basic act_ipt sch_red sch_multiq sch_prio em_meta act_police em_text sch_sfq sch_fq act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables tun mmc_block usb_storage mtk_sd mmc_core leds_gpio xhci_mtk xhci_plat_hcd xhci_pci xhci_hcd ahci libahci libata sd_mod scsi_mod gpio_button_hotplug ext4 jbd2
<4>[60140.028210]  mbcache usbcore nls_base usb_common crc32c_generic [last unloaded: xt_coova]
<4>[60140.036400] Process sh (pid: 22208, threadinfo=8da34000, task=8dcb3e80, tls=77fbbd98)
<4>[60140.044192] Stack : 00000002 00000000 8f19ca20 805abce0 00000001 81467d00 00000001 81467d00
<4>[60140.052528]         8f19c960 81467d00 8f5cdec4 811ee9bc 80640000 800d099c 8da35e08 23268b0b
<4>[60140.060861]         81467d00 8f19c960 8da35e08 800ecc88 219d1b8f 8f84ae98 81473480 00000001
<4>[60140.069214]         00000011 23268a8d 8f86244c 8da35e08 8f19c960 00000055 8dcb3e80 8da35f28
<4>[60140.077576]         8eaaea80 00000001 8eaaeabc 800f029c 00000011 00000040 00000000 8f84ae40
<4>[60140.085909]         ...
<4>[60140.088346] Call Trace:
<4>[60140.090811] [<800cf880>] __page_cache_release+0x190/0x22c
<4>[60140.096194] [<800d099c>] __put_page+0x34/0x80
<4>[60140.100540] [<800ecc88>] wp_page_copy+0x4d8/0x594
<4>[60140.105228] [<800f029c>] do_wp_page+0x7a8/0x7d4
<4>[60140.109738] [<800f1678>] handle_mm_fault+0xaa8/0xb84
<4>[60140.114703] [<8001c480>] __do_page_fault+0x120/0x478
<4>[60140.119648] [<80022128>] tlb_do_page_fault_1+0x118/0x120
<4>[60140.124935] Code: 8e030014  8e020018  02c02025 <ac620004> ac430000  8e170000  24020100  ae020014  24020200 
<4>[60140.134661] 
<4>[60140.136540] ---[ end trace 136088129c9a9746 ]---

===================================
lukasz1992 commented 5 years ago

@pparent76 Do crashes appear when you turn off WiFi? Crash logs appear to not have much common with mt76 driver; maybe it is another kernel problem.

pparent76 commented 5 years ago

I did not test with this specific kernel version, but with all versions I tested in the past, I never experienced a crash with wifi disabled.

pparent76 commented 5 years ago

I will try latests version on a wired only router to confirm that.

pparent76 commented 5 years ago

I can confirm that with wifi disabled there is no such crash in last 3 weeks. Crashes only appear when wifi is enabled.

(version: kmod-mt7603 - 4.14.87+2018-12-04-da00af00-1 )

nbd168 commented 5 years ago

Fixed in the latest version

953537867 commented 2 years ago

For me only these warnings every 3/4 days, connection is stable with 10 devices

[105622.833568] ------------[ cut here ]------------ [105622.838329] WARNING: CPU: 2 PID: 1487 at target-mipsel_24kc_musl/linux-ramips_mt7621/mt76-2021-04-11-bf45b30d/mt7603/mac.c:206 mt7603_filter_tx+0x16c/0x174 [mt7603e] [105622.853172] Modules linked in: pppoe ppp_async pppox ppp_generic mt76x2e mt76x2_common mt76x02_lib mt7603e mt76 mac80211 iptable_nat ipt_REJECT cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_MASQUERADE xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc sch_cake nf_reject_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_conntrack_rtcache nf_conncount iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat act_connmark nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 ifb mmc_block mtk_sd mmc_core leds_gpio xhci_plat_hcd xhci_pci [105622.853386] xhci_mtk xhci_hcd gpio_button_hotplug usbcore nls_base usb_common [105622.947785] CPU: 2 PID: 1487 Comm: hostapd Not tainted 5.4.111 #0 [105622.953936] Stack : 00000001 8007d738 806c0000 806beb24 80720000 806beaec 806bdc40 8ee639cc [105622.962359] 80860000 8fe5d268 80708d03 80651b4c 00000002 00000001 8ee63970 e3a64ee8 [105622.970779] 00000000 00000000 808a0000 00000000 30000000 0000015f 00000018 312e342e [105622.979197] 00000000 00001487 00000000 68203a6d 80000000 80720000 00000000 8eeb452c [105622.987615] 00000009 00000009 00000009 8eec0000 00000008 8034e8dc 00000008 80860008 [105622.996032] ... [105622.998557] Call Trace: [105623.001110] [<8000b64c>] show_stack+0x30/0x100 [105623.005643] [<805950b8>] dump_stack+0xa4/0xdc [105623.010091] [<8002bfe8>] warn+0xc0/0x10c [105623.014263] [<8002c090>] warn_slowpath_fmt+0x5c/0xac [105623.019317] [<8eeb452c>] mt7603_filter_tx+0x16c/0x174 [mt7603e] [105623.025314] [<8eeb19d4>] mt7603_sta_remove+0x94/0xfc [mt7603e] [105623.031242] [<8eea2d44>] mt76_sta_remove+0x78/0xec [mt76] [105623.036897] [<8eea2e90>] mt76_sta_state+0xd8/0x1e4 [mt76] [105623.042533] [<8e809ee4>] sta_set_sinfo+0xfac/0x102c [mac80211] [105623.048727] ---[ end trace 91b91b401c16c82b ]---

Related syslog entries for the abve:

Fri Apr 16 22:43:04 2021 daemon.notice hostapd: wlan0: AP-STA-DISCONNECTED 4c:24:98:88:f0:0d Fri Apr 16 22:43:05 2021 daemon.info hostapd: wlan0: STA 4c:24:98:88:f0:0d IEEE 802.11: authenticated Fri Apr 16 22:43:05 2021 daemon.info hostapd: wlan0: STA 4c:24:98:88:f0:0d IEEE 802.11: associated (aid 1) Fri Apr 16 22:43:05 2021 daemon.notice hostapd: wlan0: AP-STA-CONNECTED 4c:24:98:88:f0:0d Fri Apr 16 22:43:05 2021 daemon.info hostapd: wlan0: STA 4c:24:98:88:f0:0d WPA: pairwise key handshake completed (RSN) Fri Apr 16 22:43:33 2021 daemon.info hostapd: wlan0: STA d4:a3:3d:6e:98:da IEEE 802.11: authenticated Fri Apr 16 22:43:33 2021 daemon.info hostapd: wlan0: STA d4:a3:3d:6e:98:da IEEE 802.11: associated (aid 10) Fri Apr 16 22:43:33 2021 daemon.notice hostapd: wlan1: Prune association for d4:a3:3d:6e:98:da Fri Apr 16 22:43:33 2021 daemon.notice hostapd: wlan1: AP-STA-DISCONNECTED d4:a3:3d:6e:98:da Fri Apr 16 22:43:33 2021 daemon.notice hostapd: wlan0: AP-STA-CONNECTED d4:a3:3d:6e:98:da Fri Apr 16 22:43:33 2021 daemon.info hostapd: wlan0: STA d4:a3:3d:6e:98:da WPA: pairwise key handshake completed (RSN) Fri Apr 16 22:44:03 2021 daemon.info hostapd: wlan1: STA d4:a3:3d:6e:98:da IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE) Fri Apr 16 22:57:17 2021 daemon.info hostapd: wlan1: STA d4:a3:3d:6e:98:da IEEE 802.11: authenticated Fri Apr 16 22:57:17 2021 daemon.info hostapd: wlan1: STA d4:a3:3d:6e:98:da IEEE 802.11: associated (aid 2) Fri Apr 16 22:57:17 2021 daemon.notice hostapd: wlan0: Prune association for d4:a3:3d:6e:98:da Fri Apr 16 22:57:17 2021 daemon.notice hostapd: wlan0: AP-STA-DISCONNECTED d4:a3:3d:6e:98:da Fri Apr 16 22:57:17 2021 daemon.notice hostapd: wlan1: AP-STA-CONNECTED d4:a3:3d:6e:98:da Fri Apr 16 22:57:17 2021 daemon.info hostapd: wlan1: STA d4:a3:3d:6e:98:da WPA: pairwise key handshake completed (RSN) Fri Apr 16 22:57:19 2021 kern.warn kernel: [105622.833568] ------------[ cut here ]------------