Segmentation fault when trying to unload the esp32_sdio module

[Craft4Cube]

I have a custom board, with Linux SoC and ESP32-Pico. The ESP32 is connected over SDIO to the Linux SoC.

I am using ESP32 Firmware Release 1.0.2 sdio_only, and kernel 6.8.7, module is commit 6a25417fc880fd744b3b0d93c11659c3e7d86384 as used in buildroot.

Loading the module and connecting to a network seems to work fine, bluetooth also is able to discover devices.

However when trying to remove the kernel-module I get a Segmentation fault:

[   58.169902] 8<--- cut here ---
[   58.173014] Unable to handle kernel NULL pointer dereference at virtual address 00000a34 when write
[   58.182129] [00000a34] *pgd=00000000
[   58.185725] Internal error: Oops: 805 [#1] SMP ARM
[   58.190518] Modules linked in: esp32_sdio(O-)
[   58.194887] CPU: 0 PID: 198 Comm: modprobe Tainted: G           O       6.8.7 #3
[   58.202279] Hardware name: Allwinner sun8i Family
[   58.206979] PC is at hci_unregister_dev+0x44/0x158
[   58.211788] LR is at hci_unregister_dev+0x3c/0x158
[   58.216580] pc : [<c07df6d8>]    lr : [<c07df6d0>]    psr: 60030013
[   58.222842] sp : c4989eb0  ip : c3130000  fp : 004fe6bc
[   58.228062] r10: 00000081  r9 : c3130000  r8 : c1a4604c
[   58.233283] r7 : c1b3f04c  r6 : c185c700  r5 : c0f170e8  r4 : c185c000
[   58.239805] r3 : 00000000  r2 : 00000a30  r1 : 00000100  r0 : c0f170e8
[   58.246328] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   58.253459] Control: 10c5387d  Table: 41a6c06a  DAC: 00000051
[   58.259198] Register r0 information: non-slab/vmalloc memory
[   58.264863] Register r1 information: non-paged memory
[   58.269912] Register r2 information: non-paged memory
[   58.274961] Register r3 information: NULL pointer
[   58.279662] Register r4 information: slab kmalloc-8k start c185c000 pointer offset 0 size 8192
[   58.288287] Register r5 information: non-slab/vmalloc memory
[   58.293944] Register r6 information: slab kmalloc-8k start c185c000 pointer offset 1792 size 8192
[   58.302824] Register r7 information: slab kmalloc-1k start c1b3f000 pointer offset 76 size 1024
[   58.311531] Register r8 information: slab kmalloc-2k start c1a46000 pointer offset 76 size 2048
[   58.320236] Register r9 information: slab task_struct start c3130000 pointer offset 0 size 2176
[   58.328942] Register r10 information: non-paged memory
[   58.334079] Register r11 information: non-paged memory
[   58.339214] Register r12 information: slab task_struct start c3130000 pointer offset 0 size 2176
[   58.348006] Process modprobe (pid: 198, stack limit = 0x8f7f34cf)
[   58.354103] Stack: (0xc4989eb0 to 0xc498a000)
[   58.358460] 9ea0:                                     bf0089b8 c185c000 bf008644 bf000320
[   58.366633] 9ec0: bf0089b8 bf008ac8 bf008644 bf0011e4 bf008adc bf008ac8 bf008644 bf005e50
[   58.374805] 9ee0: c1b3f008 c1b3f000 bf008644 c0657a54 c1b3f008 c1a46008 bf008644 c051a75c
[   58.382978] 9f00: c1b3f008 bf008644 b6ecd178 00000081 c01002c4 c051a83c bf008644 c1105c0c
[   58.391150] 9f20: b6ecd178 c0518f90 bf008980 00000000 b6ecd178 bf006a84 bf0086c0 c018c294
[   58.399321] 9f40: 000000c0 00000000 00000000 00000000 00000000 00000000 33707365 64735f32
[   58.407492] 9f60: 00006f69 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   58.415664] 9f80: 00000000 00000000 00000000 00000000 00000000 716ab810 004ff730 0044ee5c
[   58.423837] 9fa0: 00000002 c0100060 0044ee5c 00000002 004ffd40 00000080 00000000 004ffd18
[   58.432009] 9fc0: 0044ee5c 00000002 b6ecd178 00000081 004ff920 004ff870 00000001 004fe6bc
[   58.440181] 9fe0: bec6ab20 bec6ab10 0044fc04 b6f297c0 20030010 004ffd40 00000000 00000000
[   58.448362]  hci_unregister_dev from esp_deinit_bt+0x20/0x38 [esp32_sdio]
[   58.455202]  esp_deinit_bt [esp32_sdio] from esp_remove_card+0x18/0x80 [esp32_sdio]
[   58.462893]  esp_remove_card [esp32_sdio] from esp_remove+0x80/0xf8 [esp32_sdio]
[   58.470324]  esp_remove [esp32_sdio] from sdio_bus_remove+0x30/0x124
[   58.476711]  sdio_bus_remove from device_release_driver_internal+0x184/0x1f8
[   58.483779]  device_release_driver_internal from driver_detach+0x54/0xa0
[   58.490490]  driver_detach from bus_remove_driver+0x58/0xa4
[   58.496076]  bus_remove_driver from esp_exit+0x34/0x5b0 [esp32_sdio]
[   58.502454]  esp_exit [esp32_sdio] from sys_delete_module+0x144/0x250
[   58.508923]  sys_delete_module from ret_fast_syscall+0x0/0x54
[   58.514677] Exception stack(0xc4989fa8 to 0xc4989ff0)
[   58.519731] 9fa0:                   0044ee5c 00000002 004ffd40 00000080 00000000 004ffd18
[   58.527904] 9fc0: 0044ee5c 00000002 b6ecd178 00000081 004ff920 004ff870 00000001 004fe6bc
[   58.536074] 9fe0: bec6ab20 bec6ab10 0044fc04 b6f297c0
[   58.541128] Code: e1a00005 eb0594e3 e3a01c01 e1c420d0 (e5823004) 
[   58.547317] ---[ end trace 0000000000000000 ]---

Kernel module load:

[    7.338802] mmc1: queuing unknown CIS tuple 0x01 [d9 01 ff] (3 bytes)
[    7.352894] mmc1: queuing unknown CIS tuple 0x1a [01 01 00 02 07] (5 bytes)
[    7.363027] mmc1: queuing unknown CIS tuple 0x1b [c1 41 30 30 ff ff ff ff] (8 bytes)
[    7.373913] mmc1: new SDIO card at address 0001
[    7.384148] esp32_sdio:esp_probe: ESP network device detected
[    7.390530] esp32_sdio:get_firmware_data: Rx Pre ====== 0
[    7.396040] esp32_sdio:get_firmware_data: Rx Pos ======  0
[    7.402851] esp32_sdio:get_firmware_data: Tx Pre ======  0
[    7.408385] esp32_sdio:get_firmware_data: Tx Pos ======  10
[    7.417845] esp32_sdio: probe of mmc1:0001:2 failed with error -22
[    7.664436] esp32_sdio:process_esp_bootup_event: Received ESP bootup event
[    7.671421] esp32_sdio:process_event_esp_bootup: Bootup Event tag: 3
[    7.677782] esp32_sdio:esp_validate_chipset: Chipset=ESP32 ID=00 detected over SDIO
[    7.685484] esp32_sdio:process_event_esp_bootup: Bootup Event tag: 0
[    7.691856] esp32_sdio:process_event_esp_bootup: Bootup Event tag: 4
[    7.698206] esp32_sdio:process_event_esp_bootup: Unsupported tag=4 in bootup event
[    7.705780] esp32_sdio:process_event_esp_bootup: Bootup Event tag: 1
[    7.712139] esp32_sdio:process_fw_data: ESP chipset's last reset cause:
[    7.718749] esp32_sdio:print_reset_reason: POWERON_RESET
[    7.724070] esp32_sdio:check_esp_version: ESP Firmware version: 1.0.2
[    7.730967] esp32_sdio:esp_reg_notifier: Driver init is ongoing
[    8.039497] esp32_sdio:init_bt: ESP Bluetooth init
[    8.044370] Bluetooth: Can not register HCI device
[    8.049177] esp32_sdio:print_capabilities: Capabilities: 0x1d. Features supported are:
[    8.057154] esp32_sdio:print_capabilities:    * WLAN on SDIO
[    8.062759] esp32_sdio:print_capabilities:    * BT/BLE
[    8.067809] esp32_sdio:print_capabilities:      - HCI over SDIO
[    8.073652] esp32_sdio:print_capabilities:      - BT/BLE dual mode
[   10.025963] esp32_sdio:esp_set_mac_address: 372 aa:f9:51:ae:11:67
[   15.049504] esp32_sdio:wait_and_decode_cmd_resp: Command[2] timed out
[   15.056007] esp32_sdio:cmd_set_mac: wait_and_decode_cmd_resp(priv, cmd_node) failure, ret: -22

ESP32 log:

I (73498) FW_MAIN: DEINIT Interface command

I (73500) FW_CMD: Unregistered event: 3

I (73500) wifi:flush txq
I (73500) wifi:stop sw txq
I (73502) wifi:lmac stop hw txq
I (73522) FW_MAIN: Stop Data Path

Also if i have a stream running over WiFi it seems to just hang after a few minutes, and does not allow any Network Traffic afterwards. (Ping does not work either)

Do you know what the issue could be?

[kapilkedawat]

Hi @Craft4Cube, we will check and get back..

[Craft4Cube]

Hello @kapilkedawat, any updates on this?

[kapilkedawat]

Hi @Craft4Cube, apologies for the delay, we will share the fix in some time.

[Shreyas0-7]

Hey @Craft4Cube esp_deinit_crash.zip Please apply this patch and check if you are still facing the error Steps to apply patch 1) Unzip the patch in esp_hosted directory 2) git apply esp_deinit_crash.patch

[Craft4Cube]

Hi,

your patch seems to have fixed the crash when shutting down. However now a warning about Flushing system-wide workqueues is still shown.

[   21.109948] WARNING: Flushing system-wide workqueues will be prohibited in near future.
[   21.118120] CPU: 0 PID: 233 Comm: modprobe Tainted: G           O       6.8.7 #5
[   21.125531] Hardware name: Allwinner sun8i Family
[   21.130246]  unwind_backtrace from show_stack+0x10/0x14
[   21.135512]  show_stack from dump_stack_lvl+0x68/0x74
[   21.140587]  dump_stack_lvl from esp_commands_teardown+0x38/0xc4 [esp32_sdio]
[   21.147774]  esp_commands_teardown [esp32_sdio] from esp_remove_card+0x20/0x80 [esp32_sdio]
[   21.156162]  esp_remove_card [esp32_sdio] from esp_remove+0x80/0xf8 [esp32_sdio]
[   21.163592]  esp_remove [esp32_sdio] from sdio_bus_remove+0x30/0x124
[   21.169980]  sdio_bus_remove from device_release_driver_internal+0x184/0x1f8
[   21.177047]  device_release_driver_internal from driver_detach+0x54/0xa0
[   21.183759]  driver_detach from bus_remove_driver+0x58/0xa4
[   21.189342]  bus_remove_driver from esp_exit+0x34/0x5a8 [esp32_sdio]
[   21.195719]  esp_exit [esp32_sdio] from sys_delete_module+0x144/0x250
[   21.202188]  sys_delete_module from ret_fast_syscall+0x0/0x54
[   21.207941] Exception stack(0xc4995fa8 to 0xc4995ff0)
[   21.212997] 5fa0:                   00451e5c 00000002 00502d40 00000080 00000000 00502d18
[   21.221171] 5fc0: 00451e5c 00000002 b6e7dc20 00000081 00502e58 00502870 00000001 005016bc
[   21.229340] 5fe0: bee55b60 bee55b50 00452c04 b6ee0c00

[Shreyas0-7]

wq.zip Hey @Craft4Cube can you please test the same with this patch?

[Craft4Cube]

Hey @Shreyas0-7,

Thanks for the patch. It seems that it fixed the second issue! Will you put those two patches into the source at some point, or will they stay as patches?

[Shreyas0-7]

Hey @Craft4Cube we have raised internal MR which is under review this will be merged in source

[Craft4Cube]

Hey @Shreyas0-7,

Thanks for the Information. I think we can close this as fixed for now, as it seems to work!