search

espressif / esp-hosted

Hosted Solution (Linux/MCU) with ESP32 (Wi-Fi + BT + BLE)
Other
700 stars 165 forks source link

esp-hosted-fg for esp32c2, spi master on mcu, raw throught testing is assert failed #519

Open mrzhuyuanheng opened 1 week ago

mrzhuyuanheng commented 1 week ago

Checklist

Issue or Suggestion Description

Hello,

i use ESP-Hosted-FG for esp32c2, spi master is mcu it's ok when i use it connect to wifi AP, but http download speed is very slow, about 70KB/s. i use raw througth testing, but it always assert failed as follow:

1.i want to know how to slove this issue 2.i want to know how to increase wifi speed. i try to increase CONFIG_ESP_SPI_RX_Q_SIZE and CONFIG_ESP_SPI_TX_Q_SIZE, but not work, and it always assert failed: queue_next_transaction spi_slave_api.c:502 (spi_trans->rx_buffer) or assert failed at queue_next_transaction spi_slave_api.c:722

ESP-->Host raw throught testing

esp32c2 log

*** Raw Throughput testing: ESP --> Host started ***

I (1246) NETWORK_ADAPTER: Initial set up done
I (1254) slave_ctrl: event ESPInit
I (1261) main_task: Returned from app_main()
0-1 sec       0.00 kbits/sec
1-2 sec       0.00 kbits/sec
2-3 sec       0.00 kbits/sec
3-4 sec       0.00 kbits/sec
4-5 sec       460.00 kbits/sec

assert failed: esp_spi_write spi_slave_api.c:722 (tx_buf_handle.payload)
Core  0 register dump:
MEPC    : 0x4038077e  RA      : 0x40386242  SP      : 0x3fcd1640  GP      : 0x3fcb70a8  
TP      : 0x3fcd1810  T0      : 0x37363534  T1      : 0x7271706f  T2      : 0x33323130  
S0/FP   : 0x0000007f  S1      : 0x00000001  A0      : 0x3fcd167c  A1      : 0x3fcb8ed1  
A2      : 0x00000001  A3      : 0x00000029  A4      : 0x00000001  A5      : 0x3fcbe000  
A6      : 0x0000000c  A7      : 0x76757473  S2      : 0x00000009  S3      : 0x3fcd1798  
S4      : 0x3fcb8ed0  S5      : 0x00000000  S6      : 0x00000000  S7      : 0x00000000  
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000  
T3      : 0x6e6d6c6b  T4      : 0x6a696867  T5      : 0x66656463  T6      : 0x62613938  
MSTATUS : 0x00001881  MTVEC   : 0x40380001  MCAUSE  : 0x00000002  MTVAL   : 0x00000000  
MHARTID : 0x00000000  

Stack memory:
3fcd1640: 0x40387472 0x40387502 0x3c09a5c4 0x4038cbbc 0x3fcb8b30 0x3c09a5c4 0x3fcb91b8 0x3c08b960
3fcd1660: 0x3fcb8b40 0x3fcd1674 0x3fcb8b44 0x3c08b940 0x3fcb8ed0 0x00323237 0x00000000 0x65737361
3fcd1680: 0x66207472 0x656c6961 0x65203a64 0x735f7073 0x775f6970 0x65746972 0x69707320 0x616c735f
3fcd16a0: 0x615f6576 0x632e6970 0x3232373a 0x78742820 0x6675625f 0x6e61685f 0x2e656c64 0x6c796170
3fcd16c0: 0x2964616f 0xa5a5a500 0xa5a5a5a5 0x4038ba62 0x00000000 0x00000000 0x00000000 0x400586f4
3fcd16e0: 0x3fce0000 0x00000000 0x3fcdcb70 0x400587c2 0x3fcd1810 0x400586f0 0xffffffe0 0x00000644
3fcd1700: 0x00000008 0x00000000 0x3fcbed50 0x40380b5c 0x00000008 0x00000644 0x000005c0 0x40380c6e
3fcd1720: 0x00000006 0x00000008 0x00000644 0x00000004 0x00000000 0x00000644 0x000005c0 0x40380cb8
3fcd1740: 0x00000000 0x00000000 0x00000000 0x40000000 0x00000001 0x00000640 0x00000000 0x00000000
3fcd1760: 0x00000000 0x3fcd17b4 0x00000000 0x42009a14 0x00000000 0x00000000 0x00000000 0x00000005
3fcd1780: 0x00000000 0x05c00000 0x00000000 0x00000000 0x00000000 0x00000000 0x3fcd17b4 0x420087de
3fcd17a0: 0x00000000 0x00000000 0x00000000 0x42008876 0x00000000 0x00000000 0x00000005 0x3fcbb3c4
3fcd17c0: 0x05b40000 0x00000000 0x00000000 0x02000000 0x00000000 0x00000000 0x00000000 0x40387282
3fcd17e0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fcd1800: 0x00000000 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0x00000150 0x3fcd1640 0x00000000
3fcd1820: 0x3fcb9c98 0x3fcd2198 0x3fcd1818 0x3fcb9c90 0x00000003 0x3fccf9d0 0x3fccf9d0 0x3fcd1818
3fcd1840: 0x00000000 0x00000016 0x3fcd1014 0x646e6573 0x7361745f 0x0000006b 0x00000000 0x3fcd1810
3fcd1860: 0x00000016 0x00000000 0x00000000 0x00000000 0x00000000 0x3fcbf120 0x3fcbf188 0x3fcbf1f0
3fcd1880: 0x00000000 0x00000000 0x00000001 0x00000000 0x00000000 0x00000000 0x4200413c 0x00000000
3fcd18a0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fcd18c0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fcd18e0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fcd1900: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fcd1920: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fcd1940: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fcd1960: 0x00000000 0x00000000 0x00000020 0x0060dc7c 0x00000000 0x000f4240 0x00000000 0x420093ca
3fcd1980: 0x3fcc13b8 0x00000000 0x3fcdbcbc 0x00000800 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fcd19a0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fcd19c0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fcd19e0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fcd1a00: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fcd1a20: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5

ELF file SHA256: 692a02526

Rebooting...

mcu log:

*** Booting Zephyr OS build zephyr-v3.4.0-63-g0de75cc2e493 ***
*** ListenAI CSK SDK Version: 2.0.0.0-5dc26542a9 ***
ESP peripheral capabilities: 0x3
esp: start testing of ESP->Host raw throughput
0-1 sec 0.00000 Kbits/sec
1-2 sec 0.00000 Kbits/sec
2-3 sec 0.00000 Kbits/sec
3-4 sec 0.00000 Kbits/sec
4-5 sec 0.00000 Kbits/sec

Host-->ESP raw throught testing

esp32c2 log

*** Raw Throughput testing: Host --> ESP started ***
0-1 sec       0.00 kbits/sec
1-2 sec       0.00 kbits/sec
2-3 sec       0.00 kbits/sec
4-5 sec       0.00 kbits/sec
5-6 sec       13436.56 kbits/sec
6-7 sec       16459.22 kbits/sec
7-8 sec       16459.22 kbits/sec
8-9 sec       16459.22 kbits/sec
9-10 sec       16413.59 kbits/sec
10-11 sec       16425.00 kbits/sec
W (11985) wifi:<ba-add>idx:0 (ifx:0, 74:d0:2b:4c:44:66), tid:0, ssn:0, winSize:64
11-12 sec       16447.81 kbits/sec
12-13 sec       16459.22 kbits/sec
13-14 sec       16470.62 kbits/sec
14-15 sec       16493.44 kbits/sec
15-16 sec       16447.81 kbits/sec
16-17 sec       16470.62 kbits/sec
17-18 sec       16447.81 kbits/sec
18-19 sec       16482.03 kbits/sec
19-20 sec       16459.22 kbits/sec
20-21 sec       16436.41 kbits/sec
21-22 sec       16413.59 kbits/sec
22-23 sec       16447.81 kbits/sec
23-24 sec       16447.81 kbits/sec
24-25 sec       16413.59 kbits/sec
E (26584) SPI_DRIVER: rx_pkt len+offset[16410]>max[1600], dropping it
25-26 sec       16459.22 kbits/sec
26-27 sec       16459.22 kbits/sec
27-28 sec       16482.03 kbits/sec
28-29 sec       16470.62 kbits/sec
29-30 sec       16482.03 kbits/sec
30-31 sec       16470.62 kbits/sec
31-32 sec       16425.00 kbits/sec
32-33 sec       16436.41 kbits/sec
33-34 sec       16470.62 kbits/sec
34-35 sec       16447.81 kbits/sec
35-36 sec       16436.41 kbits/sec
36-37 sec       16447.81 kbits/sec
37-38 sec       16470.62 kbits/sec
38-39 sec       16459.22 kbits/sec
39-40 sec       16447.81 kbits/sec
40-41 sec       16493.44 kbits/sec
41-42 sec       16447.81 kbits/sec
42-43 sec       16482.03 kbits/sec
43-44 sec       16459.22 kbits/sec
44-45 sec       16459.22 kbits/sec
45-46 sec       16436.41 kbits/sec
46-47 sec       16470.62 kbits/sec
47-48 sec       16482.03 kbits/sec
48-49 sec       16482.03 kbits/sec
49-50 sec       16447.81 kbits/sec
50-51 sec       16482.03 kbits/sec
51-52 sec       16436.41 kbits/sec
52-53 sec       16482.03 kbits/sec
53-54 sec       16470.62 kbits/sec
54-55 sec       16459.22 kbits/sec
E (56562) SPI_DRIVER: rx_pkt len+offset[1712]>max[1600], dropping it
55-56 sec       16459.22 kbits/sec
56-57 sec       16425.00 kbits/sec
57-58 sec       16459.22 kbits/sec
58-59 sec       16459.22 kbits/sec
59-60 sec       16459.22 kbits/sec
60-61 sec       16459.22 kbits/sec
61-62 sec       16436.41 kbits/sec
62-63 sec       16447.81 kbits/sec
63-64 sec       16447.81 kbits/sec
64-65 sec       16447.81 kbits/sec
65-66 sec       16459.22 kbits/sec
66-67 sec       16470.62 kbits/sec
67-68 sec       16459.22 kbits/sec
68-69 sec       16447.81 kbits/sec
E (70586) SPI_DRIVER: process_spi_rx: cal_chksum[86] != exp_chksum[0], drop len[6] offset[0]
69-70 sec       16447.81 kbits/sec
70-71 sec       16459.22 kbits/sec
71-72 sec       16447.81 kbits/sec
72-73 sec       16390.78 kbits/sec
E (73885) SPI_DRIVER: rx_pkt len+offset[54789]>max[1600], dropping it
73-74 sec       16470.62 kbits/sec
74-75 sec       16447.81 kbits/sec
75-76 sec       16482.03 kbits/sec
76-77 sec       16459.22 kbits/sec

my environment:

Hardware:

Software:

mantriyogesh commented 1 week ago

Esp32-c2 is low memory chipset.

If you need higher throughput might need to disable bluetooth. As much as iram to be optimised to let wifi and spi queues memory.

If you increase spi queues, it would crash when wifi back pressure is high (queue loading would exhaust memory).

If you notice the checksum errors, there is spi timing issues on your setup. You need to have 0 checksum errors.

If ESP to host throughput is not correct, please check data ready pin interrupt is fired correctly.

Handshake, data ready and reset pin are mandatory pins.

Also ensure that you are using 26MHz crystal. If you are using 40MHz crystal, software may not work correctly and also would not show any reason why it would not work.

Also disable mempool should be disabled by default to conserve some memory.

The spi frequency can go till 40MHz. But only if you have PCB. stepwise increase could be attempted.

As you get 70 KB/s, I assume your reset pin, handshake and data ready are already working. But why the ESP to host traffic is not working in that case?

If you use jumper wires, ensure to have minimal length and good contact points. At the max length 10cm. All equal.

70KBps i.e. half mbps is extremely low throughput. The throughput depends minimum of two:

  1. Transport max throughput (raw throughput)
  2. Wifi interference

Wifi max iperf UDP traffic should really reach around your transport max (little less, but just there).

Raw throughput in either direction should really be symmetrical speed in spi.

mrzhuyuanheng commented 23 hours ago

@mantriyogesh thanks for your reply!

Also ensure that you are using 26MHz crystal. If you are using 40MHz crystal, software may not work correctly and also would >not show any reason why it would not work.

i use esptool.py chip_id to get crystal is 26MHZ

  Detecting chip type... ESP32-C2
  Chip is ESP32-C2 (revision v1.2)
  Features: WiFi, BLE
  Crystal is 26MHz
  MAC: d0:ef:76:82:09:1c
  Uploading stub...
  Running stub...
  Stub running...
  Warning: ESP32-C2 has no Chip ID. Reading MAC instead.
  MAC: d0:ef:76:82:09:1c
  Hard resetting via RTS pin...

If you need higher throughput might need to disable bluetooth. As much as iram to be optimised to let wifi and spi queues memory.

my usage scenario requires both wifi and Bluetooth

Handshake, data ready and reset pin are mandatory pins.

i do use reset pin, handshake and data ready pin,

we welded the ESP8684-MINI-1 module to the PCB, also welded the ESP32C3-MINI-1 module to the other PCB, which means i test both module as spi slave. i use the same mcu host code, and esp8684 module esp32c3 module use the same esp-hosted commit code

ESP->host and host->ESP raw throughput test for ESP32C3-MINI-1 is ok.

but raw throughput test for ESP8684-MINI-1 is not ok, esp32c2 crashed when i test ESP->Host raw throughput. i can successfully run the test by modify esp32c2 esp-hosted file stats.c as follow: image

ESP->Host

esp32c2 log

  *** Raw Throughput testing: ESP --> Host started ***

  I (1047) NETWORK_ADAPTER: Initial set up done
  I (1054) slave_ctrl: event ESPInit
  I (1061) main_task: Returned from app_main()
  I (1107) slave_ctrl: Get station mac address
  I (1108) slave_ctrl: mac [d0:ef:76:82:09:1c] 
  I (1111) slave_ctrl: Get station mac address
  I (1113) slave_ctrl: mac [d0:ef:76:82:09:1c] 
  0-1 sec       0.00 kbits/sec
  1-2 sec       0.00 kbits/sec
  2-3 sec       0.00 kbits/sec
  3-4 sec       0.00 kbits/sec
  4-5 sec       11.50 kbits/sec
  5-6 sec       11500.00 kbits/sec
  6-7 sec       11500.00 kbits/sec
  7-8 sec       11500.00 kbits/sec
  8-9 sec       11500.00 kbits/sec
  9-10 sec       11500.00 kbits/sec
  10-11 sec       11500.00 kbits/sec
  11-12 sec       11500.00 kbits/sec
  12-13 sec       11500.00 kbits/sec
  13-14 sec       11500.00 kbits/sec
  14-15 sec       11500.00 kbits/sec
  15-16 sec       11500.00 kbits/sec
  16-17 sec       11500.00 kbits/sec
  17-18 sec       11500.00 kbits/sec
  18-19 sec       11500.00 kbits/sec
  19-20 sec       11500.00 kbits/sec
  20-21 sec       11500.00 kbits/sec

Host->ESP

i also modify same file stats.c on host mcu as follow, and can succefully run Host->esp test image

esp32c2 log

  1468-1469 sec       14246.41 kbits/sec
  1469-1470 sec       14212.19 kbits/sec
  1470-1471 sec       14200.78 kbits/sec
  1471-1472 sec       14223.59 kbits/sec
  1472-1473 sec       14200.78 kbits/sec
  1473-1474 sec       14212.19 kbits/sec
  1474-1475 sec       14212.19 kbits/sec
  1475-1476 sec       14212.19 kbits/sec
  1476-1477 sec       14212.19 kbits/sec
  1477-1478 sec       14223.59 kbits/sec
  1478-1479 sec       14189.38 kbits/sec
  1479-1480 sec       14200.78 kbits/sec
  1480-1481 sec       14189.38 kbits/sec
  1481-1482 sec       14212.19 kbits/sec

I tested http download again which http server in a LAN environment, esp32c2 http download average speed can up to 470KB/s. but sometimes esp32c2 would crash suddenly very infrequently

iperf test for esp32c2 always crashed, it always assert failed: queue_next_transaction spi_slave_api.c:502 (spi_trans->rx_buffer) .

I wonder if there is a stable version for esp32c2 to use?

mantriyogesh commented 5 hours ago

iperf test for esp32c2 always crashed, it always assert failed: queue_next_transaction spi_slave_api.c:502 (spi_trans->rx_buffer) .

I had observed this issue on esp32-c6. and mempool was requesting more in peak rates than expected. 50f9444c89decbc2ddb15fd8a7e1538ba6d8f881 is the fix for c6. I had not yet tested on C2 this.

But c2 was disabled for lowering code space requirement. What I would really suggest, disable tx and rx priority queues (in sdkconfig.defaults.esp32c2 to default and remove files sdkconfig and build/), as well enable mempool for c2.

I can deep down here and try these on latest master and fix, but might take some time.