search

lwfinger / rtw88

A backport of the Realtek Wifi 5 drivers from the wireless-next repo.
650 stars 184 forks source link

[rtw88_8822ce] Spontaneous Wifi adapter breakage #82

Open scrungus opened 2 years ago

scrungus commented 2 years ago

Not sure if this is the right place to put this but my wifi seems to drop out almost randomly and I have to restart my computer to fix it. Normally this is fine but now I have a process that takes about 4 days to run and is about halfway through so I can't restart.

When I do ip link set wlp1s0 up: I get 'Operation already in progress'.

When I do rmmod rtw88_8822ce rtw88_8822c rtw88_pci rtw88_core and then modprobe rtw88_8822ce it doesn't give any output, but then checking dmesg: 

[94349.904626] rtw_8822ce 0000:01:00.0: Firmware version 9.9.6, H2C version 15 

[94349.904642] rtw_8822ce 0000:01:00.0: can't change power state from D3cold to D0 (config space inaccessible)

[94349.904691] rtw_8822ce 0000:01:00.0: Firmware version 9.9.4, H2C version 15

[94349.906997] rtw_8822ce 0000:01:00.0: mac power on failed

[94349.907003] rtw_8822ce 0000:01:00.0: failed to power on mac

[94349.907004] rtw_8822ce 0000:01:00.0: failed to setup chip efuse info

[94349.907005] rtw_8822ce 0000:01:00.0: failed to setup chip information

[94349.907766] rtw_8822ce: probe of 0000:01:00.0 failed with error -114

And after doing this, my wireless interface has now disappeared. Let me know what other info I should provide. Running Ubuntu 21.04

lwfinger commented 2 years ago

My first comment would be that a code that runs for 4 days without dumping intermediate file that allows recovery is a bad design.

Your system likely needs some options set. Run the command echo "options rtw88_pci disable_msi=y disable_aspm=y" | sudo tee /etc/modprobe.d/50-rtw88_pci.conf

Then unload/reload the driver, or reboot.

scrungus commented 2 years ago

Well that may be so but I think a system that needs to be rebooted to work is also a bad design 😄

That command didn't seem to do anything, I still dont have a wireless interface. Any other ideas?

lwfinger commented 2 years ago

What messages are shown in the dmesg output when the connection drops?

After you reboot, if you do a 'sudo modprobe -rv rtw88_8822ce' then 'sudo modprobe -v rtw88_8822ce', do your options show on the rtw88_pci line.

scrungus commented 2 years ago

I'll have to post it next time it happens. It's not consistent and I don't know what the trigger is.

lwfinger commented 2 years ago

I just pushed an update to the code to bring it up to the kernel 5.18 level. I know some of the changes helped 8821ce - perhaps it will help 8822ce as well.

mcloaked commented 2 years ago

I have also had intermittent (seemingly random wifi disconnects for quite some time with 8822CE. In the journal I see lines like: May 03 19:38:01 ryzen1 kernel: rtw_8822ce 0000:01:00.0: timed out to flush pci tx ring[1] May 03 19:38:01 ryzen1 kernel: rtw_8822ce 0000:01:00.0: failed to get tx report from firmware May 03 19:38:01 ryzen1 kernel: rtw_8822ce 0000:01:00.0: timed out to flush queue 1 May 03 19:38:01 ryzen1 kernel: rtw_8822ce 0000:01:00.0: timed out to flush queue 2 May 03 19:38:01 ryzen1 kernel: rtw_8822ce 0000:01:00.0: timed out to flush pci tx ring[1] May 03 19:38:01 ryzen1 kernel: rtw_8822ce 0000:01:00.0: timed out to flush queue 1 May 03 19:38:01 ryzen1 kernel: rtw_8822ce 0000:01:00.0: timed out to flush queue 2 May 03 19:38:01 ryzen1 iwd[645]: Received Deauthentication event, reason: 4, from_ap: false

If I rmmod rmmod rtw88_8822ce and then modprobe the same module to reload it - then the wireless connection is re-esablished - my system is arch linux and the wireless set up is using systemd-networkd with iwd. Most of the time it is fine but randomly perhaps once or twice in a 5 hour period it disconnects with reason 4 as above.

The journal shows loading during boot as per:

May 03 18:26:35 ryzen1 kernel: rtw_8822ce 0000:01:00.0: enabling device (0000 -> 0003) May 03 18:26:35 ryzen1 kernel: input: HD-Audio Generic HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00> May 03 18:26:35 ryzen1 kernel: input: PC Speaker as /devices/platform/pcspkr/input/input21 May 03 18:26:35 ryzen1 kernel: rtw_8822ce 0000:01:00.0: Firmware version 9.9.4, H2C version 15 May 03 18:26:35 ryzen1 kernel: rtw_8822ce 0000:01:00.0: Firmware version 9.9.11, H2C version 15

mcloaked commented 2 years ago

Also if it is any help the modinfo shows: sudo modinfo rtw88_8822c filename: /lib/modules/5.17.5-1-stable/kernel/drivers/net/wireless/realtek/rtw88/rtw88_8822c.ko.zst license: Dual BSD/GPL description: Realtek 802.11ac wireless 8822c driver author: Realtek Corporation firmware: rtw88/rtw8822c_wow_fw.bin firmware: rtw88/rtw8822c_fw.bin srcversion: 7735D22194316D872B79644 depends: rtw88_core retpoline: Y intree: Y name: rtw88_8822c vermagic: 5.17.5-1-stable SMP preempt mod_unload sig_id: PKCS#7 signer: Build time autogenerated kernel key sig_key: 3A:EF:F8:7A:29:35:6C:02:A1:DD:D5:68:0B:64:80:01:36:06:1E:16 sig_hashalgo: sha512 signature: 30:65:02:31:00:E6:4B:AE:B3:8C:4D:EB:F2:97:7E:7E:28:73:03:87: 47:E9:AF:60:5E:3D:D1:1B:CE:4F:62:78:0F:C2:7C:9B:26:FB:58:57: 18:D7:46:20:3E:61:AC:CB:E7:A0:ED:35:BD:02:30:35:24:09:60:81: 61:E4:F0:3F:0F:EB:E8:E2:CF:85:F1:FA:7B:34:A4:50:6D:19:53:36: A2:9D:9D:0C:BA:54:1E:BE:E6:7F:EF:22:60:3A:1F:FB:32:A9:53:F9: 7D:63:5B

mcloaked commented 2 years ago

and sudo modinfo rtw88_8822ce filename: /lib/modules/5.17.5-1-stable/kernel/drivers/net/wireless/realtek/rtw88/rtw88_8822ce.ko.zst license: Dual BSD/GPL description: Realtek 802.11ac wireless 8822ce driver author: Realtek Corporation srcversion: BF9D29A6BD5A0222292439C alias: pci:v000010ECd0000C82Fsvsdbcsci alias: pci:v000010ECd0000C822svsdbcsci depends: rtw88_pci,rtw88_8822c retpoline: Y intree: Y name: rtw88_8822ce vermagic: 5.17.5-1-stable SMP preempt mod_unload sig_id: PKCS#7 signer: Build time autogenerated kernel key sig_key: 3A:EF:F8:7A:29:35:6C:02:A1:DD:D5:68:0B:64:80:01:36:06:1E:16 sig_hashalgo: sha512 signature: 30:66:02:31:00:F6:EE:FE:5F:01:3F:77:C7:CA:0A:7B:F9:C4:EA:B6: 09:E5:EA:AF:ED:25:9F:6B:87:24:CB:69:C6:49:D3:28:9A:1F:CC:3F: 5D:AD:13:A9:99:D9:7C:3F:86:84:7E:DA:FB:02:31:00:BB:2B:5D:A8: E5:5C:A7:BE:25:EC:D8:AA:9B:2B:EE:88:65:C4:7C:52:13:38:C6:22: A5:78:EA:15:21:80:1A:8A:7A:D5:4E:D6:3F:D4:89:85:AD:5B:0C:FC: 04:60:BE:1C

Kampfmoehre commented 2 years ago

I have a similar problem, since I updated via dnf (running Fedora) on 2022-07-21 Wi-Fi doesn't work most of the time. Right after boot it doesn't work until I switch to guest Wi-Fi and back. Sometimes it works nearly the full work day but sometimes it doesn't and breaks every 3-5 minutes. Strangely it seems to work longer when I leave gnome Wi-Fi settings open.

I have a HP EliteBook 855 G8 with Realtek RTL8822BE. Since I have dual boot with Windows where everything is running fine I can exclude a hardware problem.

Wi-Fi works some times then it doesn't in Fedora. When it stops working I can see this line in dmesg and jourmalctl rtw_8822ce failed to get tx report from firmware Relevant lspci lines are

01:00.0 Wireless controller [0d40]: Intel Corporation XMM7360 LTE Advanced Modem (rev 01)
02:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8822CE 802.11ac PCIe Wireless Network Adapter

and modinfo is

$ sudo modinfo rtw88_8822ce 
filename:       /lib/modules/5.18.13-200.fc36.x86_64/kernel/drivers/net/wireless/realtek/rtw88/rtw88_8822ce.ko.xz
license:        Dual BSD/GPL
description:    Realtek 802.11ac wireless 8822ce driver
author:         Realtek Corporation
alias:          pci:v000010ECd0000C82Fsv*sd*bc*sc*i*
alias:          pci:v000010ECd0000C822sv*sd*bc*sc*i*
depends:        rtw88_pci,rtw88_8822c
retpoline:      Y
intree:         Y
name:           rtw88_8822ce
vermagic:       5.18.13-200.fc36.x86_64 SMP preempt mod_unload 
sig_id:         PKCS#7
signer:         Fedora kernel signing key
sig_key:        6F:B6:86:92:50:36:0C:18:F3:98:D7:8A:3C:78:5B:67:F7:93:C3:81
sig_hashalgo:   sha256
signature:      6D:07:88:E8:CC:EE:80:8E:7A:3D:37:89:30:C0:B8:97:2E:33:A8:38:
        07:50:C8:1C:8F:00:09:E2:53:85:AF:54:35:37:9E:92:72:C6:55:30:
        FC:87:16:49:ED:39:E5:49:69:1C:1F:11:11:F5:AC:76:AF:9A:D6:D1:
        0A:A7:92:54:3D:79:9E:23:BE:2A:04:D3:37:DC:69:8A:F6:D9:05:AE:
        D5:16:A9:71:0C:96:F6:5B:74:5E:C1:56:5B:37:06:68:01:60:81:C3:
        D9:91:94:61:93:AA:25:D8:0E:84:1D:9C:21:8D:3F:7B:D5:12:51:86:
        69:B4:84:EE:39:3F:7F:9B:42:B8:52:9B:D8:6B:D8:82:9C:E5:93:4E:
        9A:C6:15:5A:73:84:EC:4F:F8:A9:68:45:79:17:23:C9:0D:29:1E:B7:
        37:A9:05:B6:BA:69:5A:AB:AA:34:FC:0B:25:4E:E6:8C:DE:C4:87:81:
        71:A3:67:2E:AE:49:37:B5:08:C3:A5:25:D4:2A:1F:8B:5A:12:50:90:
        27:B0:09:BE:9C:67:BE:2E:61:78:A3:7F:FC:2F:F7:88:DA:99:36:63:
        1E:2A:FB:09:E2:09:A6:FE:E7:94:2A:6B:CB:46:29:19:AA:CA:63:7B:
        27:D9:68:D3:65:37:CB:4B:CE:D2:3E:7F:6E:19:D1:76:35:71:8C:3A:
        8B:B8:64:6D:A1:73:DC:03:3C:1D:BA:28:F9:4F:F6:B7:B8:E2:D8:FC:
        4B:B0:17:6B:BC:F5:DE:2D:E9:57:2E:DF:07:63:E1:26:28:97:31:AE:
        8C:AD:FE:59:DE:C8:63:C6:41:AD:BC:B4:B4:AA:79:DE:29:B8:D1:9C:
        13:DD:5E:3D:CF:E0:0E:A5:D7:3D:11:0F:B5:C2:F4:DC:55:61:D3:4E:
        DF:14:EC:B7:A9:4C:86:93:04:E3:80:55:9E:56:6C:16:B1:29:6B:51:
        0D:F9:DA:7F:21:30:A7:65:BA:FC:00:24:18:6C:E6:1D:1F:1C:CF:AB:
        59:94:C0:19:DA:E6:0B:6C:DA:61:F6:35:B2:B3:50:D1:1F:B1:5B:9D:
        3C:22:3B:B8:C1:92:0D:8C:DF:0E:28:FE:57:94:C6:81:8F:88:5C:AB:
        AE:26:16:F8:AB:6F:83:5B:74:D5:16:3C:18:8A:BB:56:E1:81:B1:4E:
        9B:0A:B9:42:18:7C:30:55:51:BD:B7:ED:D8:6C:75:0B:EC:11:E7:50:
        B1:35:B0:14:59:C6:C6:8F:7B:B5:25:31:C3:CB:41:E6:FC:D7:C2:8A:
        55:7C:DE:75:F3:DF:E4:65:F7:5A:B9:CE:A4:07:D0:8C:AF:F1:2D:8C:
        AB:6D:08:74:41:05:0F:99:46:83:83:42

And these were the packages that were updated before Wi-Fi stopped working:

$ sudo dnf history info 184
Transaction ID : 184
Begin time     : Do 21 Jul 2022 07:33:53 CEST
Begin rpmdb    : 06bbc43864924b5ec38a3e6db2926d41d5d35591275f3743e07ed6d6a1f535f6
End time       : Do 21 Jul 2022 07:33:58 CEST (5 seconds)
End rpmdb      : 0e0b73cfbdae587c4a8ebc24808a759af913206647bb946f800173c400b7e137
User           : root <root>
Return-Code    : Success
Releasever     : 
Command Line   : 
Comment        : 
Packages Altered:
    Upgrade  curl-7.82.0-7.fc36.x86_64                            @updates
    Upgrade  emacs-filesystem-1:28.1-2.fc36.noarch                @updates
    Upgrade  fedora-release-common-36-18.noarch                   @updates
    Upgrade  fedora-release-identity-workstation-36-18.noarch     @updates
    Upgrade  fedora-release-workstation-36-18.noarch              @updates
    Upgrade  iwl100-firmware-39.31.5.1-136.fc36.noarch            @updates
    Upgrade  iwl1000-firmware-1:39.31.5.1-136.fc36.noarch         @updates
    Upgrade  iwl105-firmware-18.168.6.1-136.fc36.noarch           @updates
    Upgrade  iwl135-firmware-18.168.6.1-136.fc36.noarch           @updates
    Upgrade  iwl2000-firmware-18.168.6.1-136.fc36.noarch          @updates
    Upgrade  iwl2030-firmware-18.168.6.1-136.fc36.noarch          @updates
    Upgrade  iwl3160-firmware-1:25.30.13.0-136.fc36.noarch        @updates
    Upgrade  iwl3945-firmware-15.32.2.9-136.fc36.noarch           @updates
    Upgrade  iwl4965-firmware-228.61.2.24-136.fc36.noarch         @updates
    Upgrade  iwl5000-firmware-8.83.5.1_1-136.fc36.noarch          @updates
    Upgrade  iwl5150-firmware-8.24.2.2-136.fc36.noarch            @updates
    Upgrade  iwl6000-firmware-9.221.4.1-136.fc36.noarch           @updates
    Upgrade  iwl6000g2a-firmware-18.168.6.1-136.fc36.noarch       @updates
    Upgrade  iwl6000g2b-firmware-18.168.6.1-136.fc36.noarch       @updates
    Upgrade  iwl6050-firmware-41.28.5.1-136.fc36.noarch           @updates
    Upgrade  iwl7260-firmware-1:25.30.13.0-136.fc36.noarch        @updates
    Upgrade  iwlax2xx-firmware-20220708-136.fc36.noarch           @updates
    Upgrade  libappstream-glib-0.8.0-1.fc36.x86_64                @updates
    Upgrade  libcurl-7.82.0-7.fc36.x86_64                         @updates
    Upgrade  libertas-usb8388-firmware-2:20220708-136.fc36.noarch @updates
    Upgrade  linux-firmware-20220708-136.fc36.noarch              @updates
    Upgrade  linux-firmware-whence-20220708-136.fc36.noarch       @updates
    Upgrade  zenity-3.42.1-2.fc36.x86_64                          @updates
    Upgraded curl-7.82.0-6.fc36.x86_64                            @@System
    Upgraded emacs-filesystem-1:27.2-11.fc36.noarch               @@System
    Upgraded fedora-release-common-36-17.noarch                   @@System
    Upgraded fedora-release-identity-workstation-36-17.noarch     @@System
    Upgraded fedora-release-workstation-36-17.noarch              @@System
    Upgraded iwl100-firmware-39.31.5.1-135.fc36.noarch            @@System
    Upgraded iwl1000-firmware-1:39.31.5.1-135.fc36.noarch         @@System
    Upgraded iwl105-firmware-18.168.6.1-135.fc36.noarch           @@System
    Upgraded iwl135-firmware-18.168.6.1-135.fc36.noarch           @@System
    Upgraded iwl2000-firmware-18.168.6.1-135.fc36.noarch          @@System
    Upgraded iwl2030-firmware-18.168.6.1-135.fc36.noarch          @@System
    Upgraded iwl3160-firmware-1:25.30.13.0-135.fc36.noarch        @@System
    Upgraded iwl3945-firmware-15.32.2.9-135.fc36.noarch           @@System
    Upgraded iwl4965-firmware-228.61.2.24-135.fc36.noarch         @@System
    Upgraded iwl5000-firmware-8.83.5.1_1-135.fc36.noarch          @@System
    Upgraded iwl5150-firmware-8.24.2.2-135.fc36.noarch            @@System
    Upgraded iwl6000-firmware-9.221.4.1-135.fc36.noarch           @@System
    Upgraded iwl6000g2a-firmware-18.168.6.1-135.fc36.noarch       @@System
    Upgraded iwl6000g2b-firmware-18.168.6.1-135.fc36.noarch       @@System
    Upgraded iwl6050-firmware-41.28.5.1-135.fc36.noarch           @@System
    Upgraded iwl7260-firmware-1:25.30.13.0-135.fc36.noarch        @@System
    Upgraded iwlax2xx-firmware-20220610-135.fc36.noarch           @@System
    Upgraded libappstream-glib-0.7.18-4.fc36.x86_64               @@System
    Upgraded libcurl-7.82.0-6.fc36.x86_64                         @@System
    Upgraded libertas-usb8388-firmware-2:20220610-135.fc36.noarch @@System
    Upgraded linux-firmware-20220610-135.fc36.noarch              @@System
    Upgraded linux-firmware-whence-20220610-135.fc36.noarch       @@System
    Upgraded zenity-3.42.1-1.fc36.x86_64                          @@System

Note: I had a kernel update to kernel-5.18.13-200.fc36.x86_64 two days later but that had no effect on Wi-Fi.

I can provide you with more information if you need it, I need my working Wi-Fi back :-)

pickfire commented 2 years ago

For me, it's this on rtw88_8822ce. I don't used to have this issue, but lately probably due to kernel upgrade I get this since last few months (last year does not have this issue). Once or twice a day, randomly it can't use network and have to restart/reconnect the wifi, usually it happens after wake from suspend. Running arch 5.18.13-zen1-1-zen.

7月 29 08:15:48 mi kernel: rtw_8822ce 0000:01:00.0: failed to write TX skb to HCI
7月 29 08:15:48 mi kernel: rtw_8822ce 0000:01:00.0: failed to pusk skb, ret -28
7月 29 08:15:48 mi kernel: rtw_8822ce 0000:01:00.0: failed to write TX skb to HCI
7月 29 08:15:48 mi kernel: rtw_8822ce 0000:01:00.0: failed to pusk skb, ret -28
7月 29 08:15:48 mi kernel: rtw_8822ce 0000:01:00.0: failed to write TX skb to HCI
7月 29 08:15:48 mi kernel: rtw_8822ce 0000:01:00.0: failed to pusk skb, ret -28

But I did not use this repository module, I use the one that comes with kernel by default.

jvech commented 1 year ago

Here is another guy with the same problem, It happens specially when I stop using software requiring internet access for some time (about 10 mins). I've tried to remove and reload rtw88_8822c and rtw88_8822ce using insmod and rmmod, but the wifi does not get alife.

Here is some information of the kernel, the drivers and the peripherals:

$ uname -rv
6.1.57-1-lts #1 SMP PREEMPT_DYNAMIC Wed, 11 Oct 2023 05:10:50 +0000
$ lsmod | grep rtw
rtw88_8822ce           16384  0
rtw88_8822c           487424  1 rtw88_8822ce
rtw88_pci              36864  1 rtw88_8822ce
rtw88_core            278528  2 rtw88_pci,rtw88_8822c
mac80211             1310720  2 rtw88_pci,rtw88_core
cfg80211             1138688  2 rtw88_core,mac80211
$ lspci | grep -i net
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
04:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8822CE 802.11ac PCIe Wireless Network Adapter

Also when the wifi breaks the dmesg outputs some interesting messages:

[Fri Sep 29 10:05:41 2023] rtw_8822ce 0000:04:00.0: failed to send h2c command
[Fri Sep 29 10:05:41 2023] rtw_8822ce 0000:04:00.0: firmware failed to leave lps state
[Fri Sep 29 10:05:41 2023] rtw_8822ce 0000:04:00.0: failed to send h2c command
[Fri Sep 29 10:05:41 2023] rtw_8822ce 0000:04:00.0: failed to send h2c command
[Fri Sep 29 10:05:41 2023] rtw_8822ce 0000:04:00.0: failed to send h2c command
[Fri Sep 29 10:05:41 2023] ------------[ cut here ]------------
[Fri Sep 29 10:05:41 2023] WARNING: CPU: 4 PID: 24236 at drivers/net/wireless/realtek/rtw88/rtw8822c.c:4382 rtw8822c_phy_cck_pd_set+0x2d4/0x310 [rtw88_8822c]
[Fri Sep 29 10:05:41 2023] Modules linked in: snd_seq_dummy snd_hrtimer snd_seq snd_seq_device ccm ip6t_REJECT nf_reject_ipv6 xt_hl ip6_tables ip6t_rt ipt_REJECT nf_reject_ipv4 xt_LOG nf_log_syslog xt_comment nft_limit xt_limit xt_addrtype xt_tcpudp xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables libcrc32c nfnetlink nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(POE) nvidia(POE) intel_rapl_msr intel_rapl_common edac_mce_amd kvm snd_hda_codec_realtek irqbypass snd_hda_codec_generic crct10dif_pclmul crc32_pclmul snd_hda_codec_hdmi rtw88_8822ce polyval_clmulni rtw88_8822c polyval_generic rtw88_pci gf128mul rtw88_core snd_hda_intel ghash_clmulni_intel snd_intel_dspcfg sha512_ssse3 snd_intel_sdw_acpi uvcvideo btusb mac80211 btrtl snd_hda_codec btbcm videobuf2_vmalloc aesni_intel snd_hda_core videobuf2_memops btintel vfat asus_nb_wmi videobuf2_v4l2 btmtk fat snd_hwdep libarc4 crypto_simd asus_wmi snd_pcm videobuf2_common bluetooth snd_timer ledtrig_audio sp5100_tco r8169
[Fri Sep 29 10:05:41 2023]  cfg80211 videodev cryptd realtek snd mdio_devres platform_profile rapl sparse_keymap wmi_bmof pcspkr i2c_piix4 ecdh_generic mc libphy rfkill soundcore ccp k10temp i2c_hid_acpi asus_wireless i2c_hid acpi_cpufreq mousedev joydev mac_hid dm_multipath dm_mod fuse loop bpf_preload ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 usbhid amdgpu serio_raw drm_ttm_helper atkbd ttm libps2 vivaldi_fmap gpu_sched nvme drm_buddy crc32c_intel drm_display_helper nvme_core xhci_pci xhci_pci_renesas cec nvme_common video i8042 serio wmi
[Fri Sep 29 10:05:41 2023] CPU: 4 PID: 24236 Comm: kworker/u32:0 Tainted: P           OE      6.1.55-1-lts #1 2953eb1dc2513ffffee996027d73b3d9ba61d99f
[Fri Sep 29 10:05:41 2023] Hardware name: ASUSTeK COMPUTER INC. TUF Gaming FX505DT_FX505DT/FX505DT, BIOS FX505DT.316 01/28/2021
[Fri Sep 29 10:05:41 2023] Workqueue: phy0 rtw_watch_dog_work [rtw88_core]
[Fri Sep 29 10:05:41 2023] RIP: 0010:rtw8822c_phy_cck_pd_set+0x2d4/0x310 [rtw88_8822c]
[Fri Sep 29 10:05:41 2023] Code: 88 8c 18 1c 3e 00 00 48 8b 44 24 48 65 48 2b 04 25 28 00 00 00 75 44 48 83 c4 50 5b 5d 41 5c 41 5d 41 5e 41 5f e9 2c 61 d6 dc <0f> 0b eb c9 44 89 fe 48 c7 c7 18 63 90 c1 e8 b9 19 e0 db 0f 0b e9
[Fri Sep 29 10:05:41 2023] RSP: 0018:ffffb9161364fd78 EFLAGS: 00010202
[Fri Sep 29 10:05:41 2023] RAX: 00000000ffffffff RBX: 0000000000000003 RCX: 0000000000000004
[Fri Sep 29 10:05:41 2023] RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffff998884ef2080
[Fri Sep 29 10:05:41 2023] RBP: ffff998884ef208f R08: 0000000000000002 R09: 0000000000000003
[Fri Sep 29 10:05:41 2023] R10: 0000000000000035 R11: 000000000000006a R12: 0000000000000002
[Fri Sep 29 10:05:41 2023] R13: ffff998884ef2080 R14: 0000000000000003 R15: 000000000000000c
[Fri Sep 29 10:05:41 2023] FS:  0000000000000000(0000) GS:ffff998b8f300000(0000) knlGS:0000000000000000
[Fri Sep 29 10:05:41 2023] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Sep 29 10:05:41 2023] CR2: 0000560ec1dba090 CR3: 0000000106282000 CR4: 00000000003506e0
[Fri Sep 29 10:05:41 2023] Call Trace:
[Fri Sep 29 10:05:41 2023]  <TASK>
[Fri Sep 29 10:05:41 2023]  ? rtw8822c_phy_cck_pd_set+0x2d4/0x310 [rtw88_8822c aa5d4cac29d9f9eddfd9f5c6ba1584275dd521d3]
[Fri Sep 29 10:05:41 2023]  ? __warn+0x7d/0xd0
[Fri Sep 29 10:05:41 2023]  ? rtw8822c_phy_cck_pd_set+0x2d4/0x310 [rtw88_8822c aa5d4cac29d9f9eddfd9f5c6ba1584275dd521d3]
[Fri Sep 29 10:05:41 2023]  ? report_bug+0xe6/0x150
[Fri Sep 29 10:05:41 2023]  ? handle_bug+0x3c/0x80
[Fri Sep 29 10:05:41 2023]  ? exc_invalid_op+0x17/0x70
[Fri Sep 29 10:05:41 2023]  ? asm_exc_invalid_op+0x1a/0x20
[Fri Sep 29 10:05:41 2023]  ? rtw8822c_phy_cck_pd_set+0x2d4/0x310 [rtw88_8822c aa5d4cac29d9f9eddfd9f5c6ba1584275dd521d3]
[Fri Sep 29 10:05:41 2023]  rtw_phy_dynamic_mechanism+0x33c/0x6f0 [rtw88_core 224fcde0ca21d319fd6c32223fb555ae42103f8c]
[Fri Sep 29 10:05:41 2023]  rtw_watch_dog_work+0x1dc/0x250 [rtw88_core 224fcde0ca21d319fd6c32223fb555ae42103f8c]
[Fri Sep 29 10:05:41 2023]  process_one_work+0x1c7/0x3a0
[Fri Sep 29 10:05:41 2023]  worker_thread+0x51/0x390
[Fri Sep 29 10:05:41 2023]  ? process_one_work+0x3a0/0x3a0
[Fri Sep 29 10:05:41 2023]  kthread+0xde/0x110
[Fri Sep 29 10:05:41 2023]  ? kthread_complete_and_exit+0x20/0x20
[Fri Sep 29 10:05:41 2023]  ret_from_fork+0x22/0x30
[Fri Sep 29 10:05:41 2023]  </TASK>
[Fri Sep 29 10:05:41 2023] ---[ end trace 0000000000000000 ]---

@lwfinger if you need more logs or information, I will try to respond as soon as possible 🥲 .

lwfinger commented 1 year ago

First of all, read README.md. Find the section "4. Option Configuration" and follow those instructions. In your case, the options line should read:

options rtw88_core disable_lps_deep=y

jvech commented 1 year ago

Actually these logs appeared with the following parameters set on /etc/modprobe.d/50-rtw88.cfg, based on the recommendations given here. just to clarify, before setting those parameters I also had the same problem

options rtw88_core disable_lps_deep=y
options rtw88_pci disable_msi=y disable_aspm=y

Right now I'm trying the following:

  1. Disable the following modules on /etc/modprobe.d/blacklist.conf
blacklist rtw88_8822ce
blacklist rtw88_core

  1. Install the drivers from the repo following the steps of installation for all distros

  2. Set different parameter combinations for rtw_core and rtw_pci driver modules on /etc/modprobe.d/rtw.cfg, here the settings I will try.

    3.1

    rtw_core disable_lps_deep=y

    3.2

    rtw_core disable_lps_deep=y
options rtw_pci disable_msi=y disable_aspm=y

    3.3 I know is the default but, why not just to be sure?

    rtw_core disable_lps_deep=n
options rtw_pci disable_msi=n disable_aspm=n

I will be testing how it works this week. If there is a mistake or wrong assumption I'm open to comments (please guide me haha I've been trying to fix the issue for a long time :'v).

Right now my drivers are:

$ lsmod | grep rtw
rtw_8822ce             16384  0
rtw_8822c             487424  1 rtw_8822ce
rtw_pci                36864  1 rtw_8822ce
rtw_core              286720  2 rtw_8822c,rtw_pci
mac80211             1310720  2 rtw_core,rtw_pci
cfg80211             1138688  2 rtw_core,mac80211

Said that I have a question, if the answer is to obvious please send me the link or the reference to look at . But what is the difference between the previous drivers (which I assume came by default in the kernel) named as rtw88_xxxx and the ones I have right now named as rtw_xxxx?

Are they different projects?

jvech commented 1 year ago

Hi there, me again. sorry for the time of the reply, but from what I've been testing, installing and loading the module rtw_8822ce manually was enough with this set up.

rtw_core disable_lps_deep=y

I have had some kernel updates so right now my kernel is:

$ uname -rv
6.1.60-1-lts #1 SMP PREEMPT_DYNAMIC Wed, 25 Oct 2023 11:10:15 +0000

again when I loaded the module with its parameters by default:

$ grep '.*' /sys/module/rtw_*/parameters/*
/sys/module/rtw_core/parameters/debug_mask:0
/sys/module/rtw_core/parameters/disable_lps_deep:N
/sys/module/rtw_core/parameters/support_bf:Y
/sys/module/rtw_pci/parameters/disable_aspm:N
/sys/module/rtw_pci/parameters/disable_msi:N

I experienced the wifi breakage. here some logs that can help:

[ 4010.288051] ------------[ cut here ]------------
[ 4010.288059] firmware failed to ack driver for leaving Deep Power mode
[ 4010.288102] WARNING: CPU: 1 PID: 117457 at /home/jvech/Templates/rtw88/ps.c:105 rtw_power_mode_change+0xf4/0x130 [rtw_core]
[ 4010.288145] Modules linked in: snd_seq_dummy snd_hrtimer snd_seq snd_seq_device ccm ip6t_REJECT nf_reject_ipv6 xt_hl ip6_tables ip6t_rt ipt_REJECT nf_reject_ipv4 xt_LOG nf_log_syslog xt_comment nft_limit xt_limit xt_addrtype xt_tcpudp xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables libcrc32c nfnetlink nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(POE) intel_rapl_msr intel_rapl_common snd_hda_codec_realtek edac_mce_amd snd_hda_codec_generic rtw_8822ce(OE) snd_hda_codec_hdmi rtw_8822c(OE) uvcvideo kvm_amd videobuf2_vmalloc snd_hda_intel videobuf2_memops kvm rtw_pci(OE) snd_intel_dspcfg rtw_core(OE) snd_intel_sdw_acpi irqbypass btusb videobuf2_v4l2 snd_hda_codec crct10dif_pclmul nvidia(POE) btrtl crc32_pclmul videobuf2_common r8169 snd_hda_core btbcm polyval_clmulni polyval_generic gf128mul btintel snd_hwdep btmtk ghash_clmulni_intel realtek videodev hid_multitouch mac80211 asus_nb_wmi snd_pcm bluetooth asus_wmi snd_timer libarc4 sha512_ssse3 sp5100_tco
[ 4010.288247]  ledtrig_audio vfat mdio_devres fat aesni_intel crypto_simd cfg80211 cryptd snd ecdh_generic platform_profile rapl pcspkr sparse_keymap i2c_piix4 k10temp mc wmi_bmof rfkill ccp libphy soundcore asus_wireless i2c_hid_acpi i2c_hid mousedev joydev acpi_cpufreq mac_hid dm_multipath fuse dm_mod loop bpf_preload ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 usbhid amdgpu serio_raw drm_ttm_helper atkbd ttm libps2 vivaldi_fmap gpu_sched nvme drm_buddy crc32c_intel drm_display_helper nvme_core xhci_pci cec xhci_pci_renesas nvme_common i8042 video wmi serio
[ 4010.288332] CPU: 1 PID: 117457 Comm: kworker/u32:1 Tainted: P        W  OE      6.1.58-1-lts #1 80fc409a839eb045209ecf7b8a5aecc34abd74a6
[ 4010.288339] Hardware name: ASUSTeK COMPUTER INC. TUF Gaming FX505DT_FX505DT/FX505DT, BIOS FX505DT.316 01/28/2021
[ 4010.288343] Workqueue: phy0 rtw_watch_dog_work [rtw_core]
[ 4010.288372] RIP: 0010:rtw_power_mode_change+0xf4/0x130 [rtw_core]
[ 4010.288405] Code: 42 3f 44 e7 44 30 e0 78 23 45 84 ed 48 c7 c0 35 bb 7c c1 48 c7 c6 3e bb 7c c1 48 c7 c7 e0 9c 7c c1 48 0f 45 f0 e8 fc fb 4d e6 <0f> 0b 5b 5d 41 5c 41 5d e9 4f 42 44 e7 48 8b 8b 98 4b 00 00 8b 83
[ 4010.288408] RSP: 0018:ffffab29d415fdf0 EFLAGS: 00010286
[ 4010.288413] RAX: 0000000000000000 RBX: ffff9b35937420a0 RCX: 0000000000000027
[ 4010.288416] RDX: ffff9b388f261668 RSI: 0000000000000001 RDI: ffff9b388f261660
[ 4010.288419] RBP: 000003a5cc77b354 R08: 0000000000000000 R09: ffffab29d415fc68
[ 4010.288421] R10: 0000000000000003 R11: ffffffffa9acc7c8 R12: 00000000c14280ff
[ 4010.288424] R13: 0000000000000000 R14: 0000000000000000 R15: ffff9b3593746a20
[ 4010.288427] FS:  0000000000000000(0000) GS:ffff9b388f240000(0000) knlGS:0000000000000000
[ 4010.288430] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4010.288433] CR2: 00007f27e4ab4000 CR3: 0000000126820000 CR4: 00000000003506e0
[ 4010.288436] Call Trace:
[ 4010.288441]  <TASK>
[ 4010.288444]  ? rtw_power_mode_change+0xf4/0x130 [rtw_core 502729da0c707fcb3f49510eda73a1f2d7eb829c]
[ 4010.288476]  ? __warn+0x7d/0xd0
[ 4010.288486]  ? rtw_power_mode_change+0xf4/0x130 [rtw_core 502729da0c707fcb3f49510eda73a1f2d7eb829c]
[ 4010.288520]  ? report_bug+0xe6/0x150
[ 4010.288529]  ? handle_bug+0x3c/0x80
[ 4010.288537]  ? exc_invalid_op+0x17/0x70
[ 4010.288542]  ? asm_exc_invalid_op+0x1a/0x20
[ 4010.288552]  ? rtw_power_mode_change+0xf4/0x130 [rtw_core 502729da0c707fcb3f49510eda73a1f2d7eb829c]
[ 4010.288585]  rtw_pci_deep_ps+0xa7/0xd0 [rtw_pci 0ac64751eb73cfdbb7057e85792a145c9c9b68f1]
[ 4010.288599]  rtw_leave_lps+0x1d/0x1a0 [rtw_core 502729da0c707fcb3f49510eda73a1f2d7eb829c]
[ 4010.288630]  rtw_watch_dog_work+0x1d4/0x260 [rtw_core 502729da0c707fcb3f49510eda73a1f2d7eb829c]
[ 4010.288659]  process_one_work+0x1c7/0x3a0
[ 4010.288670]  worker_thread+0x51/0x390
[ 4010.288676]  ? process_one_work+0x3a0/0x3a0
[ 4010.288681]  kthread+0xde/0x110
[ 4010.288687]  ? kthread_complete_and_exit+0x20/0x20
[ 4010.288693]  ret_from_fork+0x22/0x30
[ 4010.288705]  </TASK>
[ 4010.288706] ---[ end trace 0000000000000000 ]---
[ 4010.291767] rtw_8822ce 0000:04:00.0: failed to send h2c command
[ 4010.392806] rtw_8822ce 0000:04:00.0: firmware failed to leave lps state
[ 4010.395858] rtw_8822ce 0000:04:00.0: failed to send h2c command
[ 4010.398904] rtw_8822ce 0000:04:00.0: failed to send h2c command
[ 4010.401951] rtw_8822ce 0000:04:00.0: failed to send h2c command
[ 4010.405014] rtw_8822ce 0000:04:00.0: failed to send h2c command
[ 4010.408117] rtw_8822ce 0000:04:00.0: failed to send h2c command
[ 4010.411174] rtw_8822ce 0000:04:00.0: failed to send h2c command
[ 4010.414219] rtw_8822ce 0000:04:00.0: failed to send h2c command
[ 4010.429306] ------------[ cut here ]------------

@lwfinger thanks for your help.

lwfinger commented 1 year ago

Your BIOS is messed up. These two parameters /sys/module/rtw_pci/parameters/disable_aspm:N /sys/module/rtw_pci/parameters/disable_msi:N need to be "y".