How to setup MCU host correctly?

Psychesnet commented 10 months ago

Hi guys,

My device has two chips, first one is arm with running linux 4.9 and second one is ESP32. At linux part, I succeeded insert esp_sdio driver(esp_hosted_fg version) and set it as static ip address. Default gateway is 169.254.169.1.

root@device-367b:~# ifconfig 
ethsta0   Link encap:Ethernet  HWaddr 00:01:38:9E:36:7B  
          inet addr:169.254.169.2  Bcast:169.254.169.3  Mask:255.255.255.252
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2688 (2.6 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:88 errors:0 dropped:0 overruns:0 frame:0
          TX packets:88 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:6432 (6.2 KiB)  TX bytes:6432 (6.2 KiB)

At ESP32 part, I followed document to build network_adapter project and it can run on my device. My question is Is there a doc to explain if I want to setup STA+NAT at ESP32? I expect ESP32 is STA mode which can connect to my router and ESP32 has internal IP with 169.254.169.1 to communicate with linux. Thanks.

mantriyogesh commented 10 months ago

1.

My device has two chips, first one is arm with running linux 4.9 and second one is ESP32.

Yes. This is supported in FG.

ESP32 has internal IP with 169.254.169.1 to communicate with linux

169.254.169.2 is set by you (static) or Linux default? Any address (even static) is expected to be configured by user (or use dhcp client over ethsta0)

Can you please send info to understand in detail:
- (a) Host logs:
- Linux dmesg from boot
- commands fired and their output
- (b) ESP log
- (c) git commit used at both ESP and host?
- (d) If any code changed, patch showing the change

4.

Is there a doc to explain if I want to setup STA+NAT at ESP32

We don't have supported such scenario. It is direct Wi-Fi connection to your AP. If your AP has running dhcp server, you can run dhcp client on this 'ethsta0' interface to get the IP address from your AP.
DHCP layer is above ESP-Hosted software. Sample DHCP client running is shown in https://github.com/espressif/esp-hosted/blob/master/esp_hosted_fg/docs/common/c_demo.md check 'some points to note'

I expect ESP32 is STA mode which can connect to my router

yes

Psychesnet commented 10 months ago

Hi @mantriyogesh , Thanks for quickly response.

We run ifconfig command to set static at Linux part.
I will run dhcp client to try again at Linux part and update the logs to you soon.

Have a nice day. Thanks. ^^

Psychesnet commented 10 months ago

Hi @mantriyogesh

Archive.zip Please check the attachment, it includes all changes I have. Thanks.

I think I need to explain something.

At ESP32 part, I use following code to connect fixed AP, and it works.

+    init_wifi_device();
+    wifi_config_t wifi_config = {
+        .sta = {
+            .scan_method = WIFI_ALL_CHANNEL_SCAN,
+            .sort_method = WIFI_CONNECT_AP_BY_SIGNAL,
+        },
+    };
+    const char *ssid = "EOL-0131-00";
+    const char *pass = "keepboxclosed";
+    strlcpy((char *) wifi_config.sta.ssid, ssid, sizeof(wifi_config.sta.ssid));
+    strlcpy((char *) wifi_config.sta.password, pass, sizeof(wifi_config.sta.password));
+    do_wifi_sta_connect(wifi_config);

At linux part, I run dhcp client at ethsta0 interface, looks like not working.


root@xbfc3916e-367b:~# ifconfig 
ethsta0   Link encap:Ethernet  HWaddr 00:01:38:9E:36:7B  
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:89 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:0 (0.0 B)  TX bytes:31684 (30.9 KiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:180 errors:0 dropped:0 overruns:0 frame:0 TX packets:180 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:12168 (11.8 KiB) TX bytes:12168 (11.8 KiB) root@xbfc3916e-367b:~# cat /proc/558/cmdline udhcpc-p/var/run/udhcpc-ethsta0.pid-s/lib/netifd/dhcp.script-f-t0-iethsta0-xhostname:xbfc3916e-367b-C-O121



3. esp-hosted is master branch to build the code for my esp32. Linux driver is coming from release/ng-v1.0.2 branch.
4. If it is possible, I would like to have following behavior for my device. At ESP part, it always power on, STA mode and connects to the WIFI AP. I will know the ip address for this device from WIFI AP. If user want to access the data from linux part, just connect to the ip address of this device, ESP will power on linux and bypass network packages to linux.

Have a nice day, thanks.

mantriyogesh commented 10 months ago

ESP-Hosted-FG allows to control and monitor your Wi-Fi connection right from your application on host.

DHCP cannot actually work yet as the station is not connected yet to AP, as checked from your ESP logs provided. I also have further question that your 'INIT' event from slave to host is received on SDIO yet or not.

I see the 'Start Data Path' in ESP side trace already. After this, next step was that ESP is going to send the 'INIT' event and you should receive something like:

Features supported are:
 * WLAN
 * BT/BLE
   - HCI over SDIO
   - BLE only
EVENT: 3

in your dmesg.

⚠️ If you are using different code base at ESP and Host, you will be just stuck in some undefined state. Make sure you use laster master at both ESP and host. flash code from sources.

For connection, you need to minimally run C_demo or Python_demo see embedded gif, for easy understanding. Worth to check some links:

Concentrate stepwise porting_guide for any porting needs to be done.

What is the host used? Different from Raspberry Pi? From your logs, I can see the sdio mmc1 was detected from ESP. Have you added external pull-ups? Is this SDIO connection with PCB or jumper cables? Set-UP picture if you can share?

somewhat related, but not exactly same issue: #293

mantriyogesh commented 10 months ago

I just checked you patch also. Thanks for the sharing changes you are tring.

The intention of ESP-Hosted is different. We use Host TCP-IP stack (from Linux) and the IP stack will all used from Host itself.

ESP will not at all know network layer IP/DHCP. It will just handle the Wi-Fi level handling of the packets. The way you intend to use, may not be the way ESP-Hosted is expected to be used.

Let me know if you have some specific thoughts/question about using the solution. See how the station is expected to be connected from Linux using simple python: https://github.com/espressif/esp-hosted/blob/master/esp_hosted_fg/docs/common/ctrl_path_python.gif

You can integrate C_demo or Python_demo (links above) as per your requirement and use the network interface directly from Linux host.

Psychesnet commented 10 months ago

Hi @mantriyogesh

Thanks. And yes, I never saw INIT to show up from dmesg at Linux part. Let me use same base code of ESP-Hosted to try again.

| What is the host used? Different from Raspberry Pi? -> My device is arm and router based machine. | From your logs, I can see the sdio mmc1 was detected from ESP. Have you added external pull-ups? -> No, there is no pull-up, just pin-to-pin between Linux and ESP. Is this SDIO connection with PCB or jumper cables? -> It is PCB.

At Linux part, I have 100% confidence the system is ready. Under OpenWrt framework, dhcpc or static is quite easy to setup for me. So, a question about ESP part, what msg I can see at ESP when enter INIT state or which part of the code I can print something out to trace? Thanks.

Psychesnet commented 10 months ago

Hi @mantriyogesh

After using same base code of ESP-Hosted, there is a INIT from dmesg, but still no ip to get. Any suggestion to go on? Thanks.

[    4.925199] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    4.953060] init: - preinit -
[    5.453326] ubi0: attaching mtd6
[    5.617508] ubi0: scanning is finished
[    5.638642] ubi0: attached mtd6 (name "rootfs_data", size 23 MiB)
[    5.650904] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    5.664680] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    5.678270] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[    5.692186] ubi0: good PEBs: 185, bad PEBs: 5, corrupted PEBs: 0
[    5.704281] ubi0: user volume: 1, internal volumes: 1, max. volumes count: 128
[    5.718723] ubi0: max/mean erase counter: 3/0, WL threshold: 4096, image sequence number: 1922496162
[    5.736968] ubi0: available PEBs: 0, total reserved PEBs: 185, PEBs reserved for bad PEB handling: 15
[    5.755390] ubi0: background thread "ubi_bgt0d" started, PID 82
[    6.457287] UBIFS (ubi0:0): background thread "ubifs_bgt0_0" started, PID 118
[    6.500156] UBIFS (ubi0:0): recovery needed
[    6.680245] UBIFS (ubi0:0): recovery completed
[    6.689250] UBIFS (ubi0:0): UBIFS: mounted UBI device 0, volume 0, name "rootfs_data"
[    6.704907] UBIFS (ubi0:0): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[    6.724714] UBIFS (ubi0:0): FS size: 19808256 bytes (18 MiB, 156 LEBs), journal size 1015809 bytes (0 MiB, 8 LEBs)
[    6.745386] UBIFS (ubi0:0): reserved for root: 935592 bytes (913 KiB)
[    6.758254] UBIFS (ubi0:0): media format: w4/r0 (latest is w4/r0), UUID DF08B20F-90CE-4EEB-A321-7F1EBA87F881, small LPT model
[    6.869142] block: attempting to load /tmp/ubifs_cfg/upper/etc/config/fstab
[    6.889942] block: extroot: not configured
[    6.911119] mount_root: switching to ubifs overlay
[    7.018063] procd: - early -
[    7.024142] procd: - watchdog -
[    7.672851] procd: - watchdog -
[    7.679465] procd: - ubus -
[    7.750356] procd: - init -
[    8.398048] kmodloader: loading kernel modules from /etc/modules.d/*
[    8.480435] esp32_sdio: loading out-of-tree module taints kernel.
[    8.493943] esp_reset, ESP32: Resetpin of Host is 37
[    8.493974] pinctrl_rts3915 18800000.pinctrl: request() failed for pin 37
[    8.507674] pinctrl_rts3915 18800000.pinctrl: pin-37 (18800000.pinctrl:37) status -16
[    8.545125] esp_reset, ESP32: Triggering ESP reset.
[    8.547216] kmodloader: done loading kernel modules from /etc/modules.d/*
[    9.002997] mmc1: queuing unknown CIS tuple 0x01 (3 bytes)
[    9.069167] mmc1: queuing unknown CIS tuple 0x1a (5 bytes)
[    9.084580] mmc1: queuing unknown CIS tuple 0x1b (8 bytes)
[    9.097080] mmc1: queuing unknown CIS tuple 0x80 (1 bytes)
[    9.108102] mmc1: queuing unknown CIS tuple 0x81 (1 bytes)
[    9.119123] mmc1: queuing unknown CIS tuple 0x82 (1 bytes)
[    9.131066] mmc1: queuing unknown CIS tuple 0x80 (1 bytes)
[    9.142092] mmc1: queuing unknown CIS tuple 0x81 (1 bytes)
[    9.153112] mmc1: queuing unknown CIS tuple 0x82 (1 bytes)
[    9.164190] mmc1: new SDIO card at address 0001
[    9.174197] esp_probe: ESP network device detected
[    9.188397] esp_sdio: probe of mmc1:0001:2 failed with error -22
[    9.504845] 
[    9.504845] Received INIT event from ESP32 peripheral
[    9.515057] EVENT: 2
[    9.519492] ESP slave Chip ID: 0x0
[    9.526343] EVENT: 0
[    9.530719] ESP peripheral capabilities: 0x1d
[    9.749303] Features supported are:
[    9.760740]   * WLAN
[    9.768913]   * BT/BLE
[    9.776742]     - HCI over SDIO
[    9.789461]     - BT/BLE dual mode
[    9.804752] EVENT: 3
[   34.044690] random: crng init done
[   34.051497] random: 7 urandom warning(s) missed due to ratelimiting
root@xbfc3916e-caac:~# 
root@xbfc3916e-caac:~# ifconfig 
ethsta0   Link encap:Ethernet  HWaddr A0:A3:B3:00:CA:AC  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:9968 (9.7 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:132 errors:0 dropped:0 overruns:0 frame:0
          TX packets:132 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:8904 (8.6 KiB)  TX bytes:8904 (8.6 KiB)

root@xbfc3916e-caac:~# cat /proc/548/cmdline 
udhcpc-p/var/run/udhcpc-ethsta0.pid-s/lib/netifd/dhcp.script-f-t0-iethsta0-xhostname:xbfc3916e-caac-C-O121

root@xbfc3916e-caac:~# cat /sys/bus/sdio/drivers/esp_sdio/mmc1\:0001\:1/uevent 
DRIVER=esp_sdio
SDIO_CLASS=00
SDIO_ID=6666:2222
MODALIAS=sdio:c00v6666d2222

mantriyogesh commented 10 months ago

Sorry I forgot to reply.

After init event, have you run the C_demo or python_demo to connect to AP in station mode?

Have you connected to AP before issuing dhcp request?

Psychesnet commented 10 months ago

Hi @mantriyogesh

Before trying C_demo program, I have a question. Looks like C_demo will ask ESP32 to connect an AP and device can get IP from DHCP client, am I right?

If it is true, can I implement connect AP code at ESP32 and device can get IP from DHCP client, is that possible? Thanks.

mantriyogesh commented 10 months ago

Just to add clarity, the networking (and above like DHCP) are host function.

Connect AP is kind of RPC to instruct ESP to initiate the connect. Once MAC later is established, RPC response sent from slave to host.

Host after receiving success to connect AP should ideally set the Mac address to the new virtual network interface created, ethsta0 (code already would do till this point).

The network interface is ready, so you can assign static IP or run DHCP , or something else on that network interface is upto you. The network interface scope and the network stack that is being run is at host only. Slave doesn't have network understanding as such.

You can refer to python code: https://github.com/espressif/esp-hosted/blob/504210623ec632efae1d39ad2636e235485ab3d4/esp_hosted_fg/host/linux/host_control/python_support/commands_lib.py#L106-L158

This python is simply wrapper to C. So Basically C will also have same functionality.

Psychesnet commented 10 months ago

Hi @mantriyogesh

I will try C_demo first and to see what would happen on my device. Thanks.

mantriyogesh commented 10 months ago

Okay.. sure.

I forgot the python after connect_ap is successful (from base Hosted code), tries to do dhcp at much higher layer. This is because dhcp software can be different for users.

https://github.com/espressif/esp-hosted/blob/504210623ec632efae1d39ad2636e235485ab3d4/esp_hosted_fg/host/linux/host_control/python_support/py_parse/process.py#L120-L138

In C we did not introduce this code intentionally, left to user run tthe dhcp, depending on dhcp client software.

Psychesnet commented 9 months ago

HI @mantriyogesh

I try to use C_demo to control ESP device, but there are some error.

root@3d56:~# C_demo get_sta_mac_addr
serial interface open failed, Is the driver loaded?
Failed to serial_init
init hosted control lib failed
Err Exit

This is my driver part.

root@3d56:~# cat /proc/devices 
Character devices:
  1 mem
  4 ttyS
  5 /dev/tty
  5 /dev/console
  5 /dev/ptmx
 10 misc
 13 input
 14 sound
 81 video4linux
 82 rts_camera
 89 i2c
 90 mtd
116 alsa
120 dbg_iomem
124 crypto
128 ptm
136 pts
180 usb
189 usb_device
248 rts_audio
249 ubi0
250 jpu
251 vpu
252 dma_proxy
253 watchdog
254 gpiochip

Block devices:
259 blkext
 31 mtdblock
179 mmc
254 ubiblock
root@3d56:~# lsmod 
configfs               25584  0 
esp32_sdio             18560  0

Looks like my device don't have /dev/esps0 serial device. Is there something I missing at building part? Thanks.

mantriyogesh commented 9 months ago

Can you please share:

git commit that you use at ESP and host
code patch, you have changes in both sides
Linux log dmesg from boot
Linux commands fired and their output.

Psychesnet commented 9 months ago

Hi @mantriyogesh

Looks like there is a state problem between ARM and ESP system. If I power off and power up whole system, I can see there is a esps0 device and device can get ip from dhcpc.

Received INIT event from ESP32 peripheral[    9.744000] EVENT: 2
[    9.760000] ESP slave Chip ID: 0x0
[    9.776000] EVENT: 0
[    9.788000] ESP peripheral capabilities: 0x1d
[   10.032000] Features supported are:
[   10.040000]   * WLAN
[   10.052000]   * BT/BLE
[   10.064000]     - HCI over SDIO
[   10.076000]     - BT/BLE dual mode
[   10.084000] EVENT: 3

xbfc3916e-3d56 login: root

BusyBox v1.30.1 () built-in shell (ash)

  _______                     ________        __

root@xbfc3916e-3d56:~# C_demo sta_connect
Enable heartbeat with duration 20
Heartbeat operation successful

Requested operation complete
Sleeping for some time just to showcase heartbeat
ethsta0 interface down
MAC address a0:a3:b3:00:ce:d8 set to ethsta0 interface
ethsta0 interface up

^C
Clean-up and exit
Disable Heartbeat
Heartbeat operation successful
root@xbfc3916e-3d56:~# udhcpc -i ethsta0
udhcpc: started, v1.30.1
udhcpc: sending discover
udhcpc: sending select for 192.168.99.104
udhcpc: lease of 192.168.99.104 obtained, lease time 43200
udhcpc: ifconfig ethsta0 192.168.99.104 netmask 255.255.255.0 broadcast 192.168.99.255
udhcpc: setting default routers: 192.168.99.1
ls: /etc/network.d/S*: No such file or directory
root@xbfc3916e-3d56:~# 
root@xbfc3916e-3d56:~# ping 192.168.99.1
PING 192.168.99.1 (192.168.99.1): 56 data bytes
64 bytes from 192.168.99.1: seq=0 ttl=64 time=22.135 ms
64 bytes from 192.168.99.1: seq=1 ttl=64 time=16.600 ms
64 bytes from 192.168.99.1: seq=2 ttl=64 time=24.394 ms
^C
--- 192.168.99.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 16.600/21.043/24.394 ms
root@xbfc3916e-3d56:~# ping 192.168.99.1
PING 192.168.99.1 (192.168.99.1): 56 data bytes
64 bytes from 192.168.99.1: seq=0 ttl=64 time=29.499 ms
64 bytes from 192.168.99.1: seq=1 ttl=64 time=24.454 ms
^C
--- 192.168.99.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 24.454/26.976/29.499 ms

Now, I just reboot ARM system, there is no INIT event at boot up. How can I use C_demo to control ESP go back to INIT state? Thanks.

mantriyogesh commented 9 months ago

Can you please check device tree param, broken-cd? Basically is there a way to probe again on failure?

See Porting guide: https://github.com/espressif/esp-hosted/blob/master/esp_hosted_fg/docs/Linux_based_host/porting_guide.md#14-dt-overlay

Psychesnet commented 9 months ago

Hi @mantriyogesh

broken-cd works, it fix my problem, thanks.

By the way, I want to change the source code to have following behavior, is it possible? Thanks.

After booting up, ESP32 will auto connect fixed SSID and get IP from AP.
Set static ip 192.168.1.1 at ESP side and set static ip 192.168.1.2 at Linux side, then Linux can connect to the internet. The path likes Linux(192.168.1.2) ---> ESP[192.168.1.1 <-> IP] ----> Internet

If yes, is there a doc to follow or a suggestion? Thanks.

mantriyogesh commented 9 months ago

Both (1) & (2) are not the intentions of ESP-Hosted, in fact they contradict. The intention is that ESP should only work like a co-processor and should assist in providing the Wi-Fi subsystem, whereas, the Linux or host would trigger connection any time on ESP boot with any SSID and password desired. TCP/Ip or networking stack is needed to be running on host and lower Wi-Fi LLC/MAC should be handled by co-processor.

Also, in ESP-Hosted, we intend not to buffer or store any data like SSID / password at slave, it should be driven from host. Although doesn't it fit to the intention, but you can still do it as the code is open source and you can chnage as per your wish.

In summary, Technically, (1) is still do-able, where in you just see ESP-IDF station example, you can just blindly connect to the SSID and password. In ESP-Hosted way, you can trigger the Wi-Fi connection when you receive first event after ESP bootup. (2) is possible but then, it is not ESP-Hosted ideaology. In fact, it will not be ESP-Hosted as such. Then in that case, you can actually use just iperf example from ESP-IDF and get ip connectivity. Host and ESP both can work on IP layer then any ESP-Hosted transport like SDIO/SPI would be redundant.

Psychesnet commented 9 months ago

Hi @mantriyogesh

Thanks. I think what I need is quite clear, I will study more at ESP part, like NAT function. Thanks.

espressif / esp-hosted

How to setup MCU host correctly? #305