Closed perceival closed 3 years ago
Description of the problem (how to configure, how to reproduce, how often it happens). On one R7800 after a few firmware crashes 5GHz network is not coming back. Crash frequency: Sep 19 20:40:32 router kernel: [46587.900780] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 03:21:11 router kernel: [70627.424504] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 08:46:04 ap kernel: [91324.251110] ath10k_pci 0000:01:00.0: firmware crashed! (guid n/a) Sep 20 13:31:00 ap kernel: [108420.462709] ath10k_pci 0000:01:00.0: firmware crashed! (guid n/a) Sep 20 13:54:40 router kernel: [108636.272621] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 16:01:53 ap kernel: [117472.803156] ath10k_pci 0000:01:00.0: firmware crashed! (guid n/a) Sep 20 18:43:42 router kernel: [125978.355867] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 18:49:15 router kernel: [126311.298125] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 18:57:31 router kernel: [126807.649399] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 21:30:59 router kernel: [136015.925942] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 22:04:43 router kernel: [138039.568005] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 22:23:22 router kernel: [139158.682469] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 04:24:16 router kernel: [160812.638883] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 04:35:18 router kernel: [161475.126539] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 05:00:00 router kernel: [162956.534036] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 05:33:32 router kernel: [164969.122611] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 09:41:30 ap kernel: [181049.517153] ath10k_pci 0000:01:00.0: firmware crashed! (guid n/a)
Software (OS, Firmware version, kernel, driver, etc) https://www.candelatech.com/downloads/ath10k-9984-10-4b/ath10k-fw-beta/firmware-5-ct-htt-mgt-community-qcache.bin DISTRIB_ID='OpenWrt' DISTRIB_RELEASE='SNAPSHOT' DISTRIB_REVISION='r14522-8cfb839907' DISTRIB_TARGET='ipq806x/generic' DISTRIB_ARCH='arm_cortex-a15_neon-vfpv4' DISTRIB_DESCRIPTION='OpenWrt SNAPSHOT r14522-8cfb839907' DISTRIB_TAINTS='no-all busybox'
Hardware (NIC chipset, platform, etc) Netgear R7800
Logs (dmesg, maybe supplicant and/or hostap) Sep 21 04:24:16 router kernel: [160812.638883] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 04:24:16 router kernel: [160812.638980] ath10k_pci 0001:01:00.0: qca9984/qca9994 hw1.0 target 0x01000000 chip_id 0x00000000 sub 168c:cafe Sep 21 04:24:16 router kernel: [160812.643988] ath10k_pci 0001:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0 lush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,htt-mgt-CT,set-special-CT,no-bmiss-CT,tx-rc-CT,cust-stats-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT,wmi-bcn-rc-CT crc32 9501c142 Sep 21 04:24:16 router kernel: [160812.672181] ath10k_pci 0001:01:00.0: board_file api 2 bmi_id 0:2 crc32 85498734 Sep 21 04:24:16 router kernel: [160812.693162] ath10k_pci 0001:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-file max-sta 512 raw 0 hwcrypto 1 Sep 21 04:24:16 router kernel: [160812.700678] ath10k_pci 0001:01:00.0: firmware register dump: Sep 21 04:24:16 router kernel: [160812.710734] ath10k_pci 0001:01:00.0: [00]: 0x0000000A 0x000015B3 0x00981422 0x00975B31 Sep 21 04:24:16 router kernel: [160812.716381] ath10k_pci 0001:01:00.0: [04]: 0x00981422 0x00060B30 0x0000001B 0x2E1CBC74 Sep 21 04:24:16 router kernel: [160812.724290] ath10k_pci 0001:01:00.0: [08]: 0x00000000 0x00975B31 0x00443C74 0x004070C8 Sep 21 04:24:16 router kernel: [160812.732305] ath10k_pci 0001:01:00.0: [12]: 0x00000009 0x00000000 0x009C4725 0x009C4729 Sep 21 04:24:16 router kernel: [160812.740265] ath10k_pci 0001:01:00.0: [16]: 0x00981422 0x0096FF27 0x0098EDCA 0x00000000 Sep 21 04:24:16 router kernel: [160812.748252] ath10k_pci 0001:01:00.0: [20]: 0x40981422 0x004065DC 0xFC000000 0x00420FC0 Sep 21 04:24:16 router kernel: [160812.756131] ath10k_pci 0001:01:00.0: [24]: 0x80998A56 0x0040663C 0x0040676C 0xC0981422 Sep 21 04:24:16 router kernel: [160812.764226] ath10k_pci 0001:01:00.0: [28]: 0x809CBD1D 0x0040665C 0x0042AD70 0x000EEA9C Sep 21 04:24:16 router kernel: [160812.772210] ath10k_pci 0001:01:00.0: [32]: 0x80981812 0x0040669C 0x0045686C 0x0042AD70 Sep 21 04:24:16 router kernel: [160812.780214] ath10k_pci 0001:01:00.0: [36]: 0x809C938B 0x004066DC 0x0045686C 0x0042AD70 Sep 21 04:24:16 router kernel: [160812.788180] ath10k_pci 0001:01:00.0: [40]: 0x809819D2 0x0040670C 0x0045686C 0x00009023 Sep 21 04:24:16 router kernel: [160812.796070] ath10k_pci 0001:01:00.0: [44]: 0x809CBBD4 0x004068AC 0x00000000 0x00009023 Sep 21 04:24:16 router kernel: [160812.804138] ath10k_pci 0001:01:00.0: [48]: 0x809B2A36 0x004068DC 0x009819B4 0x00427CC4 Sep 21 04:24:16 router kernel: [160812.812138] ath10k_pci 0001:01:00.0: [52]: 0x809AFCC6 0x0040691C 0x00000002 0x00427CC4 Sep 21 04:24:16 router kernel: [160812.820137] ath10k_pci 0001:01:00.0: [56]: 0x809AF407 0x0040694C 0x00406980 0x00407B88 Sep 21 04:24:16 router kernel: [160812.828103] ath10k_pci 0001:01:00.0: Copy Engine register dump: Sep 21 04:24:16 router kernel: [160812.836088] ath10k_pci 0001:01:00.0: [00]: 0x0004a000 4 4 3 3 Sep 21 04:24:16 router kernel: [160812.842358] ath10k_pci 0001:01:00.0: [01]: 0x0004a400 27 27 1 2 Sep 21 04:24:16 router kernel: [160812.848692] ath10k_pci 0001:01:00.0: [02]: 0x0004a800 8 8 71 72 Sep 21 04:24:16 router kernel: [160812.855096] ath10k_pci 0001:01:00.0: [03]: 0x0004ac00 0 0 3 0 Sep 21 04:24:16 router kernel: [160812.861681] ath10k_pci 0001:01:00.0: [04]: 0x0004b000 754 754 163 123 Sep 21 04:24:16 router kernel: [160812.868181] ath10k_pci 0001:01:00.0: [05]: 0x0004b400 0 0 255 256 Sep 21 04:24:16 router kernel: [160812.874618] ath10k_pci 0001:01:00.0: [06]: 0x0004b800 8 8 8 8 Sep 21 04:24:16 router kernel: [160812.881204] ath10k_pci 0001:01:00.0: [07]: 0x0004bc00 1 1 1 1 Sep 21 04:24:16 router kernel: [160812.887751] ath10k_pci 0001:01:00.0: [08]: 0x0004c000 0 0 127 0 Sep 21 04:24:16 router kernel: [160812.894149] ath10k_pci 0001:01:00.0: [09]: 0x0004c400 0 0 0 0 Sep 21 04:24:16 router kernel: [160812.900728] ath10k_pci 0001:01:00.0: [10]: 0x0004c800 0 0 0 0 Sep 21 04:24:16 router kernel: [160812.907262] ath10k_pci 0001:01:00.0: [11]: 0x0004cc00 0 0 0 0 Sep 21 04:24:16 router kernel: [160812.913888] ath10k_pci 0001:01:00.0: debug log header, dbuf: 0x420fc0 dropped: 0 Sep 21 04:24:16 router kernel: [160812.920353] ath10k_pci 0001:01:00.0: [0] next: 0x420fd8 buf: 0x417330 sz: 1500 len: 156 count: 6 free: 0 Sep 21 04:24:16 router kernel: [160812.927912] ath10k_pci 0001:01:00.0: ath10k_pci ATH10K_DBG_BUFFER: Sep 21 04:24:16 router kernel: [160812.937464] ath10k: [0000]: 01524BDA 10005881 00003112 0042CA94 0000000A 00000000 01524BDA 10005881 Sep 21 04:24:16 router kernel: [160812.943454] ath10k: [0008]: 00003112 0042CA94 0000000E 00000001 01524C50 1400581D 00000000 0044FE68 Sep 21 04:24:16 router kernel: [160812.952924] ath10k: [0016]: 000EC9C0 00000006 00000000 01524C50 1000581B 0000DE50 00000000 00000000 Sep 21 04:24:16 router kernel: [160812.962016] ath10k: [0024]: 000EC9C0 01524C53 14006402 71103332 B0EBF384 000050DE 0042AD70 0000000A Sep 21 04:24:16 router kernel: [160812.971139] ath10k: [0032]: 01524C53 17FC0001 00981422 000015B3 000015B3 004064CC 91104569 Sep 21 04:24:16 router kernel: [160812.980258] ath10k_pci 0001:01:00.0: ATH10K_END
just found qcache firmware info statin fiemware is nof yet ready to be installed. closin the issue
Description of the problem (how to configure, how to reproduce, how often it happens). On one R7800 after a few firmware crashes 5GHz network is not coming back. Crash frequency: Sep 19 20:40:32 router kernel: [46587.900780] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 03:21:11 router kernel: [70627.424504] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 08:46:04 ap kernel: [91324.251110] ath10k_pci 0000:01:00.0: firmware crashed! (guid n/a) Sep 20 13:31:00 ap kernel: [108420.462709] ath10k_pci 0000:01:00.0: firmware crashed! (guid n/a) Sep 20 13:54:40 router kernel: [108636.272621] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 16:01:53 ap kernel: [117472.803156] ath10k_pci 0000:01:00.0: firmware crashed! (guid n/a) Sep 20 18:43:42 router kernel: [125978.355867] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 18:49:15 router kernel: [126311.298125] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 18:57:31 router kernel: [126807.649399] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 21:30:59 router kernel: [136015.925942] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 22:04:43 router kernel: [138039.568005] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 20 22:23:22 router kernel: [139158.682469] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 04:24:16 router kernel: [160812.638883] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 04:35:18 router kernel: [161475.126539] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 05:00:00 router kernel: [162956.534036] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 05:33:32 router kernel: [164969.122611] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 09:41:30 ap kernel: [181049.517153] ath10k_pci 0000:01:00.0: firmware crashed! (guid n/a)
Software (OS, Firmware version, kernel, driver, etc) https://www.candelatech.com/downloads/ath10k-9984-10-4b/ath10k-fw-beta/firmware-5-ct-htt-mgt-community-qcache.bin DISTRIB_ID='OpenWrt' DISTRIB_RELEASE='SNAPSHOT' DISTRIB_REVISION='r14522-8cfb839907' DISTRIB_TARGET='ipq806x/generic' DISTRIB_ARCH='arm_cortex-a15_neon-vfpv4' DISTRIB_DESCRIPTION='OpenWrt SNAPSHOT r14522-8cfb839907' DISTRIB_TAINTS='no-all busybox'
Hardware (NIC chipset, platform, etc) Netgear R7800
Logs (dmesg, maybe supplicant and/or hostap) Sep 21 04:24:16 router kernel: [160812.638883] ath10k_pci 0001:01:00.0: firmware crashed! (guid n/a) Sep 21 04:24:16 router kernel: [160812.638980] ath10k_pci 0001:01:00.0: qca9984/qca9994 hw1.0 target 0x01000000 chip_id 0x00000000 sub 168c:cafe Sep 21 04:24:16 router kernel: [160812.643988] ath10k_pci 0001:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0 lush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,htt-mgt-CT,set-special-CT,no-bmiss-CT,tx-rc-CT,cust-stats-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT,wmi-bcn-rc-CT crc32 9501c142 Sep 21 04:24:16 router kernel: [160812.672181] ath10k_pci 0001:01:00.0: board_file api 2 bmi_id 0:2 crc32 85498734 Sep 21 04:24:16 router kernel: [160812.693162] ath10k_pci 0001:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-file max-sta 512 raw 0 hwcrypto 1 Sep 21 04:24:16 router kernel: [160812.700678] ath10k_pci 0001:01:00.0: firmware register dump: Sep 21 04:24:16 router kernel: [160812.710734] ath10k_pci 0001:01:00.0: [00]: 0x0000000A 0x000015B3 0x00981422 0x00975B31 Sep 21 04:24:16 router kernel: [160812.716381] ath10k_pci 0001:01:00.0: [04]: 0x00981422 0x00060B30 0x0000001B 0x2E1CBC74 Sep 21 04:24:16 router kernel: [160812.724290] ath10k_pci 0001:01:00.0: [08]: 0x00000000 0x00975B31 0x00443C74 0x004070C8 Sep 21 04:24:16 router kernel: [160812.732305] ath10k_pci 0001:01:00.0: [12]: 0x00000009 0x00000000 0x009C4725 0x009C4729 Sep 21 04:24:16 router kernel: [160812.740265] ath10k_pci 0001:01:00.0: [16]: 0x00981422 0x0096FF27 0x0098EDCA 0x00000000 Sep 21 04:24:16 router kernel: [160812.748252] ath10k_pci 0001:01:00.0: [20]: 0x40981422 0x004065DC 0xFC000000 0x00420FC0 Sep 21 04:24:16 router kernel: [160812.756131] ath10k_pci 0001:01:00.0: [24]: 0x80998A56 0x0040663C 0x0040676C 0xC0981422 Sep 21 04:24:16 router kernel: [160812.764226] ath10k_pci 0001:01:00.0: [28]: 0x809CBD1D 0x0040665C 0x0042AD70 0x000EEA9C Sep 21 04:24:16 router kernel: [160812.772210] ath10k_pci 0001:01:00.0: [32]: 0x80981812 0x0040669C 0x0045686C 0x0042AD70 Sep 21 04:24:16 router kernel: [160812.780214] ath10k_pci 0001:01:00.0: [36]: 0x809C938B 0x004066DC 0x0045686C 0x0042AD70 Sep 21 04:24:16 router kernel: [160812.788180] ath10k_pci 0001:01:00.0: [40]: 0x809819D2 0x0040670C 0x0045686C 0x00009023 Sep 21 04:24:16 router kernel: [160812.796070] ath10k_pci 0001:01:00.0: [44]: 0x809CBBD4 0x004068AC 0x00000000 0x00009023 Sep 21 04:24:16 router kernel: [160812.804138] ath10k_pci 0001:01:00.0: [48]: 0x809B2A36 0x004068DC 0x009819B4 0x00427CC4 Sep 21 04:24:16 router kernel: [160812.812138] ath10k_pci 0001:01:00.0: [52]: 0x809AFCC6 0x0040691C 0x00000002 0x00427CC4 Sep 21 04:24:16 router kernel: [160812.820137] ath10k_pci 0001:01:00.0: [56]: 0x809AF407 0x0040694C 0x00406980 0x00407B88 Sep 21 04:24:16 router kernel: [160812.828103] ath10k_pci 0001:01:00.0: Copy Engine register dump: Sep 21 04:24:16 router kernel: [160812.836088] ath10k_pci 0001:01:00.0: [00]: 0x0004a000 4 4 3 3 Sep 21 04:24:16 router kernel: [160812.842358] ath10k_pci 0001:01:00.0: [01]: 0x0004a400 27 27 1 2 Sep 21 04:24:16 router kernel: [160812.848692] ath10k_pci 0001:01:00.0: [02]: 0x0004a800 8 8 71 72 Sep 21 04:24:16 router kernel: [160812.855096] ath10k_pci 0001:01:00.0: [03]: 0x0004ac00 0 0 3 0 Sep 21 04:24:16 router kernel: [160812.861681] ath10k_pci 0001:01:00.0: [04]: 0x0004b000 754 754 163 123 Sep 21 04:24:16 router kernel: [160812.868181] ath10k_pci 0001:01:00.0: [05]: 0x0004b400 0 0 255 256 Sep 21 04:24:16 router kernel: [160812.874618] ath10k_pci 0001:01:00.0: [06]: 0x0004b800 8 8 8 8 Sep 21 04:24:16 router kernel: [160812.881204] ath10k_pci 0001:01:00.0: [07]: 0x0004bc00 1 1 1 1 Sep 21 04:24:16 router kernel: [160812.887751] ath10k_pci 0001:01:00.0: [08]: 0x0004c000 0 0 127 0 Sep 21 04:24:16 router kernel: [160812.894149] ath10k_pci 0001:01:00.0: [09]: 0x0004c400 0 0 0 0 Sep 21 04:24:16 router kernel: [160812.900728] ath10k_pci 0001:01:00.0: [10]: 0x0004c800 0 0 0 0 Sep 21 04:24:16 router kernel: [160812.907262] ath10k_pci 0001:01:00.0: [11]: 0x0004cc00 0 0 0 0 Sep 21 04:24:16 router kernel: [160812.913888] ath10k_pci 0001:01:00.0: debug log header, dbuf: 0x420fc0 dropped: 0 Sep 21 04:24:16 router kernel: [160812.920353] ath10k_pci 0001:01:00.0: [0] next: 0x420fd8 buf: 0x417330 sz: 1500 len: 156 count: 6 free: 0 Sep 21 04:24:16 router kernel: [160812.927912] ath10k_pci 0001:01:00.0: ath10k_pci ATH10K_DBG_BUFFER: Sep 21 04:24:16 router kernel: [160812.937464] ath10k: [0000]: 01524BDA 10005881 00003112 0042CA94 0000000A 00000000 01524BDA 10005881 Sep 21 04:24:16 router kernel: [160812.943454] ath10k: [0008]: 00003112 0042CA94 0000000E 00000001 01524C50 1400581D 00000000 0044FE68 Sep 21 04:24:16 router kernel: [160812.952924] ath10k: [0016]: 000EC9C0 00000006 00000000 01524C50 1000581B 0000DE50 00000000 00000000 Sep 21 04:24:16 router kernel: [160812.962016] ath10k: [0024]: 000EC9C0 01524C53 14006402 71103332 B0EBF384 000050DE 0042AD70 0000000A Sep 21 04:24:16 router kernel: [160812.971139] ath10k: [0032]: 01524C53 17FC0001 00981422 000015B3 000015B3 004064CC 91104569 Sep 21 04:24:16 router kernel: [160812.980258] ath10k_pci 0001:01:00.0: ATH10K_END