al177
commented
2 years ago Make sure you have CONFIG_CFG80211 and CONFIG_MAC80211 in the kernel config.
Closed YusufYYK closed 1 year ago
al177
commented
2 years ago Make sure you have CONFIG_CFG80211 and CONFIG_MAC80211 in the kernel config.
YusufYYK
commented
2 years ago Hi,
After kernel rebuild(5.4.209 with your instruction) the device works. I can connect to network and ping the router. But after some time it always occurs a problem and wlan0 is disappeared. Until the restart again module work. How can i fix it?
64 bytes from 192.168.88.1: icmp_seq=289 ttl=64 time=5.43 ms 64 bytes from 192.168.88.1: icmp_seq=290 ttl=64 time=5.52 ms 64 bytes from 192.168.88.1: icmp_seq=291 ttl=64 time=6.89 ms 64 bytes from 192.168.88.1: icmp_seq=292 ttl=64 time=5.45 ms 64 bytes from 192.168.88.1: icmp_seq=293 ttl=64 time=5.84 ms 64 bytes from 192.168.88.1: icmp_seq=294 ttl=64 time=5.38 ms 64 bytes from 192.168.88.1: icmp_seq=295 ttl=64 time=6.21 ms 64 bytes from 192.168.88.1: icmp_seq=296 ttl=64 time=5.43 ms 64 bytes from 192.168.88.1: icmp_seq=297 ttl=64 time=1.78 ms 64 bytes from 192.168.88.1: icmp_seq=298 ttl=64 time=5.41 ms 64 bytes from 192.168.88.1: icmp_seq=299 ttl=64 time=5.35 ms 64 bytes from 192.168.88.1: icmp_seq=300 ttl=64 time=4.91 ms 64 bytes from 192.168.88.1: icmp_seq=301 ttl=64 time=5.99 ms 64 bytes from 192.168.88.1: icmp_seq=302 ttl=64 time=1.95 ms 64 bytes from 192.168.88.1: icmp_seq=303 ttl=64 time=5.36 ms 64 bytes from 192.168.88.1: icmp_seq=304 ttl=64 time=95.4 ms
---------- dmesg----------
[ +1,320559] wlan0: authenticate with 98:0d:67:15:6d:47
[ +0,005315] wlan0: send auth to 98:0d:67:15:6d:47 (try 1/3)
[ +0,010860] wlan0: authenticated
[ +0,012267] wlan0: associate with 98:0d:67:15:6d:47 (try 1/3)
[ +0,011682] wlan0: RX AssocResp from 98:0d:67:15:6d:47 (capab=0x431 status=0 aid=3)
[ +0,007939] wlan0: associated
[ 597.844199] esp_sdio_remove enter
[ 597.848717] sif_disable_irq release irq failed
[ 597.853185] ------------[ cut here ]------------
[ 597.857929] WARNING: CPU: 0 PID: 360 at kernel/workqueue.c:3049 flush_work+0xac/0xb8
[ 597.865884] Modules linked in: esp8089(O)
[ 597.869911] CPU: 0 PID: 360 Comm: kworker/0:0 Tainted: G O 5.4.209 #1
[ 597.877651] Hardware name: Allwinner sun8i Family
[ 597.882365] Workqueue: events_freezable mmc_rescan
[ 597.887182] [
al177
commented
2 years ago That's often a sign that the driver is not asserting the correct reset GPIO or the reset line isn't toggling. Does this happen if you rmmod then insmod the driver without a reboot?
If wlan0 is disappearing, then it also may be an SDIO signal integrity issue.
YusufYYK
commented
2 years ago Hi,
I am using breadboard and esp8266-201(removed spi flash) kit with allwinner v3s(licheepi zero board). VCC pins connected to upper power line and GNDs are connected to bottom line of breadboard. After using osciloscope, i see that when the wpa_supplicant connect to network, the power line pk to pk voltage drops reach 800 mV and very noisy. I connect Vcc and Gnd closely on same line and add bypass caps. And add pull up 10k to SDIO pins , 330R series resistors. I checked whole circuit with osciloscope, every thing is good. I use trigger function to catch the reset pin but when error occurs, nothing change on reset pin. I thing it is a soft issue. Something cause to remove sdio. Just insmod and rmmod result is below paste.Same result with rmmod. What is your opinion? How can i fix it?
64 bytes from 192.168.88.1: icmp_seq=871 ttl=64 time=2.49 ms
64 bytes from 192.168.88.1: icmp_seq=872 ttl=64 time=7.16 ms
64 bytes from 192.168.88.1: icmp_seq=873 ttl=64 time=1.68 ms
[ 985.843772] esp_sdio_remove enter
[ 985.848314] sif_disable_irq release irq failed
[ 985.852780] ------------[ cut here ]------------
[ 985.857515] WARNING: CPU: 0 PID: 337 at kernel/workqueue.c:3049 flush_work+0xac/0xb8
[ 985.865467] Modules linked in: esp8089(O)
[ 985.869495] CPU: 0 PID: 337 Comm: kworker/0:1 Tainted: G O 5.4.209 #1
[ 985.877235] Hardware name: Allwinner sun8i Family
[ 985.881951] Workqueue: events_freezable mmc_rescan
[ 985.886763] [
root@PI:~# insmod esp8089.ko [ 81.190402] esp8089: loading out-of-tree module taints kernel. [ 81.216053] [ 81.216053] EAGLE DRIVER VER:bdf5087c3deb [ 81.216053] [ 81.224242] ESP8089 reset via GPIO 132 [ 81.228072] sun8i-v3s-pinctrl 1c20800.pinctrl: 1c20800.pinctrl supply vcc-pe not found, using dummy regulator [ 81.364097] mmc1: card 0001 removed [ 82.532864] mmc1: queuing unknown CIS tuple 0x01 (3 bytes) [ 82.546548] mmc1: queuing unknown CIS tuple 0x1a (5 bytes) [ 82.556171] mmc1: queuing unknown CIS tuple 0x1b (8 bytes) [ 82.563025] mmc1: queuing unknown CIS tuple 0x80 (1 bytes) [ 82.569967] mmc1: queuing unknown CIS tuple 0x81 (1 bytes) [ 82.576609] mmc1: queuing unknown CIS tuple 0x82 (1 bytes) [ 82.582172] mmc1: new SDIO card at address 0001 [ 82.595836] esp_sdio_dummy_probe enter [ 82.813529] esp_sdio_init power up OK [ 83.297330] esp_host:bdf5087c3deb [ 83.297330] esp_target: e826c2b3c9fd 57 18202 [ 83.297330] [ 83.306708] esp_readwrite_file: file /system/lib/modules/test_results filp_open error [ 83.316249] first normal exit [ 83.319488] esp_sdio_remove enter [ 83.329763] sif_disable_irq release irq failed [ 83.453807] eagle_sdio: probe of mmc1:0001:1 failed with error -110 root@PI:~# [ 83.604033] mmc1: card 0001 removed [ 83.657802] mmc1: queuing unknown CIS tuple 0x01 (3 bytes) [ 83.671553] mmc1: queuing unknown CIS tuple 0x1a (5 bytes) [ 83.680243] mmc1: queuing unknown CIS tuple 0x1b (8 bytes) [ 83.688123] mmc1: queuing unknown CIS tuple 0x80 (1 bytes) [ 83.694828] mmc1: new SDIO card at address 0001 [ 84.079137] esp_host:bdf5087c3deb [ 84.079137] esp_target: e826c2b3c9fd 57 18202 [ 84.079137]
root@PI:~# rmmod esp8089.ko
[ 93.721339] esp_sdio_exit
[ 93.724382] esp_sdio_remove enter
[ 93.730355] ------------[ cut here ]------------
[ 93.735102] WARNING: CPU: 0 PID: 274 at kernel/workqueue.c:3049 flush_work+0xac/0xb8
[ 93.743023] Modules linked in: esp8089(O-)
[ 93.747165] CPU: 0 PID: 274 Comm: rmmod Tainted: G O 5.4.209 #1
[ 93.754386] Hardware name: Allwinner sun8i Family
[ 93.759113] [
root@PI:~#
YusufYYK
commented
2 years ago I'm using driver with change gpio in sdio_stub.c
/ reset GPIO parameter defaults to GPIO 0 (ID_SD) on the Raspberry Pi / static int esp_reset_gpio = 132; module_param(esp_reset_gpio, int, 0); MODULE_PARM_DESC(esp_reset_gpio, "ESP8089 CH_PD reset GPIO number");
YusufYYK
commented
2 years ago I am using wpa_supplicant with external driver. Kernel compiled with "cfg80211 wireless extensions compatibility" option. otherwise i can connect with nl80211 driver but after 2 seconds connection drops. insmod esp8089.ko wpa_supplicant -B -D wext -i wlan1 -c /etc/wpa_supplicant/wpa_supplicant.conf dhclient wlan0
al177
commented
2 years ago 800mV drop on the supply is enough that I'd throw a cap near the ESP module.
But more worrying is the breadboard. I'm confident your problem is signal integrity. Look through the closed bugs here. A fair number of them were cleared up by shortening and equalizing length of the wires. I can't speak for your SoC, but you may get better results with your setup if you can turn down the SDIO clock speed significantly.
YusufYYK
commented
2 years ago Hi Andrew,
I focused on SI. And short connections with pullup and series resistors.i am sure about SI. After connect the board to my wifi network, i noticed that pinging from other computer to the board "No error occurs"! When i ping with ssh or serial connection inside the board to any device same error occurs. How can it be? is it kernel issue or something else? What is your opinion? Any advice?
YusufYYK
commented
2 years ago i try download a file with big size(standby time: 1 hour). No problem occurs!
Connecting to mirror.ufs.ac.za (196.255.240.40:80) saving to 'linuxmint-14-kde-dvd-64bit.iso' linuxmint-14-kde-dvd 100% |****| 1045M 0:00:00 ETA 'linuxmint-14-kde-dvd-64bit.iso' saved
YusufYYK
commented
2 years ago After that again ping fails.
64 bytes from 192.168.88.150: seq=179 ttl=64 time=2.721 ms
64 bytes from 192.168.88.150: seq=180 ttl=64 time=5.508 ms
64 bytes from 192.168.88.150: seq=181 ttl=64 time=5.657 ms
64 bytes from 192.168.88.150: seq=182 ttl=64 time=5.611 ms
64 bytes from 192.168.88.150: seq=183 ttl=64 time=6.098 ms
64 bytes from 192.168.88.150: seq=184 ttl=64 time=1.914 ms
64 bytes from 192.168.88.150: seq=185 ttl=64 time=5.782 ms
[ 3259.168114] esp_sdio_remove enter
[ 3259.172651] sif_disable_irq release irq failed
[ 3259.177117] ------------[ cut here ]------------
[ 3259.181854] WARNING: CPU: 0 PID: 496 at kernel/workqueue.c:3049 __flush_work+0xac/0xb8
[ 3259.189808] Modules linked in: esp8089(O)
[ 3259.193834] CPU: 0 PID: 496 Comm: kworker/0:1 Tainted: G O 5.4.209 #1
[ 3259.201574] Hardware name: Allwinner sun8i Family
[ 3259.206287] Workqueue: events_freezable mmc_rescan
[ 3259.211102] [
al177
commented
2 years ago It still looks like an SI issue. There are a bunch of things you need to check for: reflections (usually fixed w/ small value resistor on data and clock lines near the ESP), stubs (breadboard contacts make for large stubs), crosstalk, bus clock rate, and skew from clock to each data line (all wires between the Allwinner SoC and ESP must be the same length). From a hardware perspective, I'd try turning the clock rate as low as possible for the Allwinner's SDIO interface and see if that clears things up, then go from there.
esp_sdio_remove()is hooked to the remove callback for the driver. The SDIO driver will assume the device is dead and remove the device if it times out on a transfer that didn't complete.
You should also set up SDIO tracing before installing the module and bringing up the interface to see why it's being removed by the kernel. This is a good guide for the debug process: https://community.silabs.com/s/article/linux-sdio-detection?language=en_US
YusufYYK
commented
2 years ago I use another software fping(open source project) then error occured but module is not removed and kernel is not crashed. It is smarter then classical ping software. It continue pinging after error. So I will buy ESP-12F and try again. As you said it is SI problem. I thing this ESP-201 modules board SI is not good. After trying with 12F i will feedback. Thanks
192.168.88.1 : [375], 64 bytes, 1.70 ms (6.43 avg, 1% loss) 192.168.88.1 : [376], 64 bytes, 5.49 ms (6.43 avg, 1% loss) 192.168.88.1 : [377], 64 bytes, 95.4 ms (6.67 avg, 1% loss) 192.168.88.1 : [378], 64 bytes, 5.47 ms (6.67 avg, 1% loss) 192.168.88.1 : [379], 64 bytes, 5.55 ms (6.66 avg, 1% loss) 192.168.88.1 : [380], 64 bytes, 5.40 ms (6.66 avg, 1% loss) [ 419.208929] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.215614] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.221984] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.228381] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.234746] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.241075] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.247425] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.304810] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.311220] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.317634] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.324017] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 419.330349] sunxi-mmc 1c10000.mmc: data error, sending stop command 192.168.88.1 : [381], 64 bytes, 5.40 ms (6.66 avg, 1% loss) 192.168.88.1 : [382], 64 bytes, 8.02 ms (6.66 avg, 1% loss) 192.168.88.1 : [383], 64 bytes, 7.70 ms (6.66 avg, 1% loss) 192.168.88.1 : [384], 64 bytes, 5.53 ms (6.66 avg, 1% loss) 192.168.88.1 : [385], 64 bytes, 8.19 ms (6.66 avg, 1% loss) 192.168.88.1 : [386], 64 bytes, 2.51 ms (6.65 avg, 1% loss) 192.168.88.1 : [387], 64 bytes, 5.39 ms (6.65 avg, 1% loss)
YusufYYK
commented
2 years ago Hi Andrew,
i try with ESP-12F and error occurs again. I try with different routers and result is same. I changed SDIO interface speed , try with 25 MHz and 50 Mhz, result is same . After them, i noticed that when i decrease period of ping no error occurs.
ping 192.168.88.1 -i 0.01 (10 ms no error) (tried with 25 Mhz and 50 Mhz sdio speed) ping 192.168.88.1 -i 0.1 (bigger than 10ms error occurs always) (speed is not important, error occurs)
--> reached 65535 seq with no error (10 ms period) 64 bytes from 192.168.88.1: icmp_seq=65534 ttl=64 time=1.39 ms 64 bytes from 192.168.88.1: icmp_seq=65535 ttl=64 time=1.54 ms 64 bytes from 192.168.88.1: icmp_seq=0 ttl=64 time=1.48 ms 64 bytes from 192.168.88.1: icmp_seq=1 ttl=64 time=1.40 ms 64 bytes from 192.168.88.1: icmp_seq=2 ttl=64 time=1.43 ms
This is not SI problem. Maybe kernel, module driver or tools problem. I haven't enough knowledge to detect the soft point. Did you make test with ping(or any ICMP packet tool) on RPI ? Maybe same issue have on the RaspberryPI.
YusufYYK
commented
2 years ago I try it with kernel 4.10.15 again same result.
YusufYYK
commented
2 years ago I use script for be sure about ping tool. It close ping after finish and starts new process. Same result. It isn't about ping buffer or process number.
for i in $(seq 5000) do ping 192.168.88.1 -c 1 sleep 1 echo $i done
YusufYYK
commented
2 years ago It is most likely module driver issue. I think it is a bug. This bug attempt to remove driver from kernel.
al177
commented
2 years ago I dusted off my test setup (Pi2B, ESP12F on my WiFi Pants HAT) and attempted to reproduce your results. The Pi is running the latest kernel (5.15.56) with a build from the HEAD of this project (with a minor fix to remove now-unsupported set/get_fs() calls and module filesystem access). Both ping intervals (100ms, 10ms) ran for extended periods of time without error. I also ran your ping loop for about 1000 iterations without fail. Finally I used iperf for bandwidth measurements with no issues with a 9.63MB payload with a performance of 7.8Mbits/s (AP is across the house).
During the above tests I did not see the kernel errors that you are reporting.
Without being able to reproduce this on the platform I have at hand, I'm out of suggestions for you. We can keep this open for a few months, and if you make any progress please update this issue and submit any PRs if you do find that a fix for the driver is needed.
YusufYYK
commented
2 years ago Ok. Can you share fixed driver for kernel 5.15.56? I will test in my board again.if it fails, i will change my SoC and try with RPI or different board. I will share results after tests.
al177
commented
2 years ago I have committed the changes and am testing a new .deb package. If you're building from source you can pull the latest.
I don't think this will change anything for you as the difference won't be ifdef'd in unless you're using 5.10.0 or later
YusufYYK
commented
2 years ago I try with 5.15.56 and same result with ping tool. With iperf result is good. I see "sunxi-mmc 1c10000.mmc: data error, sending stop command" error but it doesn't make kernel corrupt like fping. As a result only one thing remains, is about board(licheepi) SI or SoC. I will ignore this issue. If it is about SI when i design a board it will fixed. Thank you very much for your interest Andrew.
Connecting to host 192.168.88.146, port 5201 [ 5] local 192.168.88.95 port 59926 connected to 192.168.88.146 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.68 MBytes 14.1 Mbits/sec 0 124 KBytes [ 5] 1.00-2.00 sec 954 KBytes 7.82 Mbits/sec 12 106 KBytes [ 5] 2.00-3.00 sec 1.30 MBytes 10.9 Mbits/sec 0 129 KBytes [ 5] 3.00-4.00 sec 1.30 MBytes 10.9 Mbits/sec 3 102 KBytes .. .. [ 5] 694.00-695.00 sec 1.30 MBytes 10.9 Mbits/sec 0 144 KBytes [ 5] 695.00-696.00 sec 1.12 MBytes 9.38 Mbits/sec 0 173 KBytes [ 5] 696.00-697.00 sec 817 KBytes 6.70 Mbits/sec 0 221 KBytes [ 5] 697.00-698.00 sec 1.39 MBytes 11.7 Mbits/sec 0 221 KBytes [ 5] 698.00-699.00 sec 607 KBytes 4.97 Mbits/sec 45 115 KBytes [ 1211.148209] sunxi-mmc 1c10000.mmc: data error, sending stop command [ 5] 699.00-700.00 sec 1.55 MBytes 13.0 Mbits/sec 0 129 KBytes [ 5] 700.00-701.00 sec 1.12 MBytes 9.40 Mbits/sec 0 137 KBytes [ 5] 701.00-702.00 sec 1.12 MBytes 9.38 Mbits/sec 0 139 KBytes [ 5] 702.00-703.00 sec 1.30 MBytes 10.9 Mbits/sec 0 140 KBytes
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-752.01 sec 863 MBytes 9.63 Mbits/sec 1167 sender [ 5] 0.00-752.01 sec 0.00 Bytes 0.00 bits/sec receiver
YusufYYK
commented
1 year ago Hi friend,
I solve the crash problem. It is about sleep mode. When i set no_auto_sleep=1 on config of your driver the problem is gone away.
al177
commented
1 year ago It's interesting that we don't see that behavior on the Raspberry Pi platforms. Maybe there's an edge case in this driver or the platform's SDIO driver that only shows on some platforms.
For now I'm going to close this and consider reopening if we run into this again. Glad you got it working!
Hi,
When i compile module with kernel 5.3.5 this error occurs on stage 2. How can i fix it? In this web page these errors given as warning. (https://whycan.com/t_4326.html)
ubuntu@ubuntu:~/Desktop/pi/esp8089$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- -C ../linux-5.3.5/ M=$(pwd) modules make: Entering directory '/home/ubuntu/Desktop/pi/linux-5.3.5' CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_debug.o CC [M] /home/ubuntu/Desktop/pi/esp8089/sdio_sif_esp.o CC [M] /home/ubuntu/Desktop/pi/esp8089/spi_sif_esp.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_io.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_file.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_main.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_sip.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_ext.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_ctrl.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_mac80211.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_utils.o CC [M] /home/ubuntu/Desktop/pi/esp8089/esp_pm.o CC [M] /home/ubuntu/Desktop/pi/esp8089/testmode.o LD [M] /home/ubuntu/Desktop/pi/esp8089/esp8089.o Building modules, stage 2. MODPOST 1 modules ERROR: "ieee80211_rx_napi" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_queue_work" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_hdrlen" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_beacon_get_tim" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_unregister_hw" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_scan_completed" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "wiphy_rfkill_start_polling" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_stop_queues" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_ready_on_channel" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_tx_status" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_alloc_hw_nm" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_wake_queues" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_remain_on_channel_expired" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_stop_tx_ba_cb_irqsafe" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "wiphy_rfkill_set_hw_state" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_get_buffered_bc" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_register_hw" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_start_tx_ba_session" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_free_hw" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_start_tx_ba_cb_irqsafe" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! ERROR: "ieee80211_stop_tx_ba_session" [/home/ubuntu/Desktop/pi/esp8089/esp8089.ko] undefined! make[1]: [scripts/Makefile.modpost:103: modules-modpost] Error 1 make: [Makefile:1630: modules] Error 2 make: Leaving directory '/home/ubuntu/Desktop/pi/linux-5.3.5'
Thanks,