Unclear usability of `feature/esp_as_mcu_host` (EHM-13)

m4nko commented 7 months ago

Hi, first of all: thank you for your work!

Now that this is done: I'm unsure about how to use this branch for building host and slave parts. Slave part builds OK, idf.py menuconfig et. al. works as expected.

Now, host part seems to be not buildable (I'm also not well versed in CMake, so apologies if I missed something obvious). A idf.py menuconfig in host/port/examples/wifi/softAP complains about missing include directories esp, netif, ... (the new directory structure is totally different from esp-hosted ).

I've read some issues (#311, esp. this comment ) so that breakage may be intended because this is a work in progress?

If not, how to build the host part?

Thanks!

mantriyogesh commented 6 months ago

Sorry for the late reply..

Yes this branch is in development. We made sure that ESP is correctly working with very stable throughout, in both spi and sdio case.

May we know which MCU are you using?

m4nko commented 6 months ago

May we know which MCU are you using?

I plan to use an STMH7 single core with SPI transport.

The ESP-as-host variant shall act as reference implementation to check against.

As this branch is in development, is it unsuited to be used as base for ESP-as-host? Like I wrote in the first post, I don't seem to be able to configure the host part of the project.

mantriyogesh commented 6 months ago

ESP as host - stability

Oh no, It works just perfect as one ESP as slave and one ESP as host. Currently, Slave ESP : ESP32/ESP32-C2/C3/S2/S3/C6 Host ESP: ESP32/ESP32-C2/C3/S2/S3/H2/P4

are working fine.

Why existing code failing to build for non-ESP MCU

Actually for host code, we just try to re-use as much as ESP-IDF elements as anyway ESP can re- use the esp_events etc components directly from ESP-IDF.

I had tried the port layer for STM32 with separating some components are defining & integrating them locally for non-ESP host.

STM32 as port

I had tried to port this for STM32F469NIH6U earlier & patch is 0001-STM32-SPI-port-for-STM32F4.patch over base Hocted commit, https://github.com/espressif/esp-hosted/commit/504f4f203c27d5a9e5e3bb1b34f7101e43cd8f59

Please refer readme.txt file once patch is applied. The 'ioc' file is STM32F4 specific, you might want port the ioc file for your STM32H7. 'IOC' file is compact way of configuration of storing project settings.

If the patch method is confusing for you, I had also made tar bundle of code & full workspace, if that helps. But anyway, if port the IOC file from example, it should be more natural way to use.

Full-code: esp_hosted_feature_branch_stm_port.tgz

Workspace-copy: workspace_19mar24.tgz

Known issues

The throughput was less around 4Mbps, but the reason was that the STM32 host was wasting the clock a lot. STM32-F4 did not have multiple FIFO settings as such, whereas STM32-H7 may support '16_data_size' FIFO) So Potentially H7 would have higher data rate.

To be honest, This porting I would not term perfect as it just gets you basic connectivity & needs further work to be done. But it is worth to try it out, if it suites you.

STM32 as host is tricky to maintain

The problem with STM32 is that the HAL and the config/capabilities change a lot from one STM32XX to other. Which cannot be much generic. Despite this, we would add sample STM32 as host in our example to port the STM32 as a new host similar to the patch shared above.

Please do let us know if you are able to port and build for H7.

m4nko commented 6 months ago

Thank you for the comprehensive answer and for the example files.

Sorry for being dense, but I think my problem is more primitive: I have checked out the branch feature/esp_as_mcu_host.

Now:

PS C:\code\esp_as_host\slave> idf.py set-target esp32c3
Adding "set-target"'s dependency "fullclean" to list of commands with default set of options.
Executing action: fullclean
Build directory 'C:\code\esp_as_host\slave\build' not found. Nothing to clean.
Executing action: set-target
Set Target to: esp32c3, new sdkconfig will be created.
Running cmake in directory C:\code\esp_as_host\slave\build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=c:\opt\espressif\tools\python_env\idf5.3_py3.12_env\Scripts\python.exe -DESP_PLATFORM=1 -DIDF_TARGET=esp32c3 -DCCACHE_ENABLE=1 C:\code\esp_as_host\slave"...
*************************************************************************************
                    Building ESP-Hosted-FG Firmware version :: 0.0.5
*************************************************************************************
-- Existing sdkconfig 'C:/code/esp_as_host/slave/sdkconfig' renamed to 'C:/code/esp_as_host/slave/sdkconfig.old'.
-- Found Git: C:/opt/bin/PortableGit/bin/git.exe (found version "2.44.0.windows.1")
...

and then it builds.

What should I do for host?

I will try out the STM32F469 variant with the files you provided.

mantriyogesh commented 6 months ago

Actually, if feasible, if you have two ESP devices, you can try ESP to ESP first on feature/esp_as_mcu_host branch. That should give you correct possible achievement and expectations that stm32 can reach.

I think you can go through documentation documentation from master branch (do not follow, just for reading) https://github.com/espressif/esp-hosted/blob/master/esp_hosted_fg/docs/MCU_based_host/Getting_started.md#getting-started-with-mcu-based-host to make you understand how the ioc file is used.

The problem with master is that the control path design is old and in-line with Linux, which did not suite ESP as host, we aimed to support every function call as rpc in esp-idf Wifi library. With new control path, it give tremendous flexibility to esp as host (and MCU as host) user with ready examples.

In new branch including control path change, we have added a lot of fixes, lwip functionality, bluetooth functionality etc, which lacks in master.

Once you read the master document, please refer to readme.txt created in top folder (once you had applied the patch)

It is essentially, two things:

Importing project using ioc in new stm32cubeide workspace, then existing the workspace
running prepare_project.sh (in your case windows, need to update prepare_project.bat similar to prepare_project.sh) as mentioned in readme.txt, reopen the workspace in step (1), clean and rebuild project

Configuration are handled easy in the new branch using esp_hosted_config.h file.

Let me know if you can get to this point, else I will formulate step by step instructions, to get you onboard easily

mantriyogesh commented 6 months ago

Do you have STM32F469 with you?

m4nko commented 6 months ago

Sorry for the long pause, but work got busy.

Actually, if feasible, if you have two ESP devices, you can try ESP to ESP first on feature/esp_as_mcu_host branch. That should give you correct possible achievement and expectations that stm32 can reach.

Yes, that is my reasoning also. I want to use the two ESPs as reference.

In new branch including control path change, we have added a lot of fixes, lwip functionality, bluetooth functionality etc, which lacks in master.

That is the reason why I want to try out the branch.

Do you have STM32F469 with you?

Yes, I do have one. I also have some STM32H7 from Weact-Studio.

I will have to pause for a short while. But I will post here with updates.

Thanks again for all your work!

mantriyogesh commented 6 months ago

Ok sure. Please do let us know as you to it or stuck anywhere.

gawied commented 6 months ago

Hi,

The ESP-as-host part seems not to be buildable - seems the main project, that includes that component, is missing. Any help here?

Thanks

m4nko commented 6 months ago

Hi,

The ESP-as-host part seems not to be buildable - seems the main project, that includes that component, is missing. Any help here?

Thanks

This is my initial impression. But then I pivoted to the STM-ESP setup to check (~~with mild success, it compiles, but the logs show errors...~~ that's on me, I made a mistake when I initially used the files provided by @mantriyogesh. But following the readme.txt from the sources, I could get the example with STM32F469 to work. Thanks)

If there is a working ESP(host)+ESP(slave) SoftAP example (+ lwIP woult be cherry on top) with the new feature/esp_as_mcu_host branch, that would be fantastic.

m4nko commented 6 months ago

Actually, if feasible, if you have two ESP devices, you can try ESP to ESP first on feature/esp_as_mcu_host branch. That should give you correct possible achievement and expectations that stm32 can reach.

@mantriyogesh Could you explain in more detail how this can be achieved? I think the objection from @gawied is valid.

mantriyogesh commented 5 months ago

Hello @gawied , @m4nko ,

We have working ESP as slave and ESP as host solution, but it is little dependent on esp-idf older & non public branch. We will provide detailed steps to get it working with latest master on GitHub esp-idf next week.

So that you can evaluate and let us know the feedback.

m4nko commented 5 months ago

We will provide detailed steps to get it working with latest master on GitHub esp-idf next week.

Much appreciated, thanks!

m4nko commented 5 months ago

Hello @gawied , @m4nko ,

We have working ESP as slave and ESP as host solution, but it is little dependent on esp-idf older & non public branch. We will provide detailed steps to get it working with latest master on GitHub esp-idf next week.

So that you can evaluate and let us know the feedback.

Any news? I've checked new commits in esp-idf and in esp-hosted but could not find related info.

Thanks!

mantriyogesh commented 5 months ago

@SohKamYung-Espressif , Can you please state the steps for one ESP as host MCU?

SohKamYung-Espressif commented 5 months ago

@m4nko The "ESP as slave and ESP as host solution" now can be found at ESP Registry (https://components.espressif.com/components/espressif/esp_hosted), which should make it easier to get up and running. We are currently using it with the ESP-IDF iperf example (in examples/wifi/iperf).

I have tested it with a ESP32 (the hosted master) connected using SPI to a ESP32-C6 (the hosted slave).

Here are the steps to setup:

On the Host side (ESP32):

apply components.txt to your ESP-IDF. If should patch examples/wifi/iperf/main/idf_component.yml to add the required components for ESP Hosted on the host side: components.txt
by default, ESP chips have built-in Wi-Fi. This has to be disabled to allow ESP-Hosted to provide the Wi-Fi interface. See soc.txt, which disables Wi-Fi for the ESP32. Use it as a reference to disable Wi-Fi on other ESP SOCs. soc.txt

In Menuconfig, the configuration for ESP-Hosted can be found at Component config ---> ESP-Hosted config. Using SPI as the Hosted interface is easier than SDIO, if you prototype using wires to connect the ESP Hosted host and slave. Build the host application as per normal (idf.py set-target, idf.py menuconfig, idf.py flash).

On the Slave side (ESP32-C6):

check out the ESP Hosted example from the Components Registry into an empty directory: idf.py create-project-from-example "espressif/esp_hosted^0.0.5-dev.16:slave". (Note: the version number may change.)

In Menuconfig, the configuration for ESP-Hosted slave can be found at Example Configuration. Make sure both Hosted host and slave use the correct SPI settings and GPIOs. Build and flash the slave application as per normal.

You should now be able to run the ESP-IDF iperf example via Hosted.

For better iperf performance, you can look at this file to configure network and buffer settings in your sdk.defaults.<esp chip>: performance.txt

Let me know if you have more questions on the ESP to ESP setup using ESP-Hosted.

SohKamYung-Espressif commented 5 months ago

@m4nko Here is some additional information to help you get started with ESP-IDF and ESP Hosted:

The ESP-IDf Get Started Guide (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/index.html) which goes through installing the ESP-IDF source code and tools.

Once the ESP-IDF tools have been setup, you can use it as follows:

idf.py set-target <soc> tells the IDF which soc (esp32, esp32c6, etc.) to build the code for
idf.py menuconfig runs Menuconfig which is use to configure the parameters for the build
idf.py build to build the code
idf.py -p <port> flash to flash the code where is the serial port the ESP device can be found
idf.py -p <port> monitor to run the terminal to get debug messages from the ESP device connected at
For ESP Hosted, various GPIOs for SPI have to be connected together to work. It is recommended to use short wires, 5 - 10 cm in length (shorter is better) to minimise propagation delay and noise. You can also start with lower SPI CLK values (5-10 Mhz) to verify that the connects are correct and stable.

Here is a small example table showing the GPIOs used to connect a ESP32 (Host) to ESP32-C6/C2/C3 slave. If you are using another ESP chip as host or slave, let me know and I can provide the recommended GPIOs to use:

GPIO	ESP32-C6/C2/C3 (Slave)	ESP32 (Host)
MOSI	7	13
MISO	2	12
CLK	6	14
CS	10	15
Handshake	3	26
Data Ready	4	4
Reset	RST	5

mantriyogesh commented 5 months ago

Supported slaves: ESP32, ESP32-C2/C3/C6, S2,S3 Supported hosts: ESP32, ESP32-C2/C3/C6, S2,S3, H2, P4

Features: Wifi with LWIP, Classic BT, BLE Transports: SPI, SDIO, UART

m4nko commented 5 months ago

If you are using another ESP chip as host or slave, let me know and I can provide the recommended GPIOs to use

I have two C3s that I will try with.

EDIT: With the two C3s the iperf example runs well (SPI with the default configs). Thanks for your assistance.

gawied commented 5 months ago

I could get iperf running with a S3 as host and a C6 as slave. Got about 18 Mbits/sec throughput with SPI default configuration. Thanks for your detailed instructions.

m4nko commented 5 months ago

Glad, that it worked out well for two ESPs.

Shall I close the issue and open a new one for other porting endeavors?

mantriyogesh commented 5 months ago

Yes that works be great

espressif / esp-hosted-mcu