Espressif ESP Architecture Support

[nashif]

As a developer I would like to be able to develop using ESP32 chips so that Zephyr can be run on LX6 CPUs.

ESP32 is a dual-core system with two Harvard Architecture Xtensa LX6 CPUs from Espressif

+Acceptance Criteria+

• ESP32 development boards can flash and run Zephyr
• Latest Zephyr boots
• Architecture configuration supported in Zephyr Project
• Open-source toolchain from Espressif Supported in Zephyr Project
• Drivers created, e.g. SPI, I2C, I2S, DAC, ADC, etc. (see related stories)

[nashif]

by Siwei Xu:

Good idea! I have a ESP32 DevKitC board, and I also want to porting zephyr to it.

[nashif]

by Siwei Xu:

BTW, support for ESP8266 is also a good idea I think.

[nashif]

by Robert Beatty:

Siwei Xu , added ZEP-2059 for ESP8266. I'd like to see ESP32 in v1.9.

[nashif]

by Linh Nguyen:

I'd like to see ESP family soon also. I will try to provide some code. Hopefully. Thanks.

[nashif]

by Siwei Xu:

Leandro Pereira , There are already has xtensa-toolchain under the zephyr SDK, but I don't sure it's fully support ESP32/ESP8266 SoC. So, should we need to add the espressif released toolchain to zephyr SDK ?

[nashif]

by Robert Beatty:

Current Xtensa-toolchain does not fully support ESP32/ESP8266.

When talking with Cadence, they pointed us to Espressif's open source toolchain and suggested that we look into those tools.

[nashif]

by Siwei Xu:

Leandro Pereira , I compared xtensa context related code under zephyr and esp-idf(espressif released SDK). I found some differences that, esp-idf's xtensa context implemented as multi-processor supported.

and from the esp32 technical reference manual, we know that esp32 has dual CPU cores and a series of CPU specific interrupts.

so I think we can not use the existed xtensa code under arch/xtensa for esp32 directly.

[nashif]

by Leandro Pereira:

We were able to build Zephyr using the Espressif SDK, and we're working on getting Zephyr to boot using it.

[nashif]

by Robert Beatty:

Leandro Pereira

Awesome job. Great progress.

[nashif]

by Siwei Xu:

good job!

[nashif]

by Ivan Grokhotkov:

Hi all, we (Espressif) are going to run the current code and see if we can help with the bringup of Zephyr on ESP32.

Seeing that SMP is not supported in Zephyr yet, we could possibly stick with the single-core version of context switching code. That would allow, for example, to run Zephyr on one core, and the low-level (non-opensource) part of our WiFi stack on the other one.

[nashif]

by Anas Nashif:

Ivan Grokhotkov Thank you for commenting here. We are planning to use the ESP-32 for enabling SMP support in Zephyr once we have the basic architecture support and the board booting our kernel.

[nashif]

by Robert Beatty:

Ivan Grokhotkov ,

Good to see Espressif here. Leandro Pereira and Rajavardhan Gundi have been investigating ESP32 support--be great to get in touch with them.

[nashif]

by Leandro Pereira:

Ivan Grokhotkov , there's an [experimental branch on my GitHub|https://github.com/lpereira/zephyr/tree/esp32-hack].

It builds Zephyr with Espressif's toolchain and relies on some things from esp-idf. It almost boots, but crashes right after jumping into Zephyr's kernel entry point (_Cstart). Some guidance or insight to have a basic version running on this board would be appreciated.

[nashif]

by Ivan Grokhotkov:

Leandro Pereira Thanks for pointing to your experimental branch. I had some progress with it today.

First, a few notes about the startup flow on the ESP32. At reset, ESP32 starts executing from _ResetVector located in ROM code (at 0x40000400). Execution proceeds to the ROM startup code, where depending on the state of the bootstrapping pins, either the application is loaded from flash, or the bootloader enters UART "download mode". This download mode allows the host to program new application into the flash memory over UART, and is the usual way of loading programs into the ESP32. You may find the description of the startup flow here: https://docs.espressif.com/idf/en/latest/api-guides/general-notes.html#first-stage-bootloader

If you use gdb to load program into RAM, you need to consider what initialization steps have been performed by the ROM code and which still need to be performed by the program.

If the ROM code has not run at all, i.e. you have done (in gdb)

mon reset halt
load
x $pc=_start
c

then _ResetVector has not had a chance to run yet, so some initialization has not been performed yet.

This could be fixed by setting $pc=_ResetVector, but for some reason the generated elf file doesn't include the reset vector... although it does include other vectors.

Because _ResetVector was not available in the generated elf file, I tried doing the initialization another way — let the ROM code do all the initialization, then perform a break, load Zephyr into RAM, and set PC to _start. In other words,

# connect GPIO0 to GND — this tells the bootloader to enter download mode (i.e. so it sits idle, and doesn't try loading application from Flash), then do external reset so that the bootstrapping pins are latched properly
# reset without halting (ignore gdb's warning after 'c'):
mon reset
c
# wait a second, then break
<Ctrl-c>
# at this point we are in some idle loop inside ROM code; load new program
load
$pc = _start
c

This almost works, the only remaining problem is that the VECBASE still points to 0x40000000 (i.e. vectors in ROM). As a quick hack I have modified arch/xtensa/core/crt1.S a bit, and added

movi a2, 0x40080000
wsr a2, vebase

before the "CALL _Cstart".

With this change, the program doesn't crash, and if I break it after some time, the PC is in k_cpu_idle, which sounds reasonable.

Now, obviously the above procedure is very much a hack and we should consider something different, going forward. For the initial tests, when the program is loaded via gdb into RAM, I suggest adding _ResetVector into the generated elf file, and marking it as the entry point. (EDIT: created a PR which adds back missing ResetVector: https://github.com/lpereira/zephyr/pull/1)

Eventually, once we start running code from flash (which is needed for any non-trivial program), we will have to let the ROM bootloader perform initialization. We also need to generate the binary in the format which ESP32's ROM bootloader can recognize. We have a python tool which does that (https://github.com/espressif/esptool), but I don't know enough about Zephyr yet to understand what is the right way to integrate such vendor tools into the build process.

[nashif]

by Ivan Grokhotkov:

Another note: I see that the current tree is introducing a SoC called "LX6" and a board called "esp32". I don't know if there will be other LX6-based SoCs supported by Zephyr, but I think that making "esp32" a "soc" itself would make sense, because ESP32 will need a lot of soc-specific startup code to be useful. Peripherals, linker script, and memory map are all specific to the ESP32, and will not be the same if support for another LX6-based SoC is added. It would also make sense if SMP is to be supported eventually, because LX6 by itself is a single CPU.

[nashif]

by Anas Nashif:

I don't know if there will be other LX6-based SoCs supported by Zephyr, but I think that making "esp32" a "soc" itself would make sense, because ESP32 will need a lot of soc-specific startup code to be useful.

Agree on that. In xtensa things are different from other architectures.

[nashif]

by Paul Sokolovsky:

How it's different? Xtensa LX6 is conceptually about the same notion as ARM Cortex-M0, well, maybe Cortex-M0+. Nothing's new under the sun ;-).

[nashif]

by Anas Nashif:

How it's different? Xtensa LX6 is conceptually about the same notion as ARM Cortex-M0, well, maybe Cortex-M0+. Nothing's new under the sun

For starters... https://ip.cadence.com/ipportfolio/tensilica-ip/xtensa-customizable

[nashif]

by Steve Williams:

LX6 is a processor generator platform, each processor is customized by the SoC developer. LX6 spans ARM Cortex M0 to R4/5 in capability, depending on caches and local memories and pipeline configuration. There are also DSP instruction set modules for imaging, audio, radar, neural net, and vector floating point, plus users can add their own instructions and customer register sets and datatypes. Anyway that's enough marketing. The point is that "LX6" can mean a lot of different things, the name "LX6" is not a unique identifier, best not to use it in SoC or Board naming.

ESP32 is an IC with processors and I/Os inside, wouldn't that make it an SoC System on Chip? Boards made using the SoC have other names. I have one called "Feather HUZZAH" from Adafruit.

ESP32 .

[nashif]

by Paul Sokolovsky:

Anas Nashif

For starters... https://ip.cadence.com/ipportfolio/tensilica-ip/xtensa-customizable

Nice marketing read, so what? Every CPU architecture is "customizable", but it's usually a headache to manage and support those "customizations", the more kudos goes to Tensilica for marketizing the opposite and to Cadence for not killing off the idea.

But back to Zephyr. It has support for x86, ARC, ARM, Nios, RISC-V, and suddenly... Xtensa... is... different? Umm, maybe no.

In Linux, ARM arch was "different" for a long time: https://www.gadgetdaily.xyz/linus-torvalds-threatens-to-cut-off-arm/ . Hope Xtensa won't take that place in Zephyr ;-).

[nashif]

by Anas Nashif:

Paul Sokolovsky In Zephyr Cortex-M0+ is not defined as an SoC, cortex-m0+ is a core that is part of an SoC that is vendor specific, an SoC in Zephyr is part of a SoC series that is part of an SoC family, so ESP32 fits just find as an SoC under the LX6 family or if LX6 is the core, an SoC with LX6 core. Not sure what issue exactly. This is just the same thing we have it for all other architectures unless I am missing something. I guess the way things are done with Xtensa are slightly different and that is completely fine, we should be able to fit this in Zephyr. There is no expectation that all architectures be the same in how things are organised or structured. X86 is different, so is ARC.

[nashif]

by Leandro Pereira:

Could someone (maybe Robert Beatty ?) please clarify what "Latest Zephyr boots and is fully functional" (from the acceptance criteria) actually means in terms of hardware support? This will help me prioritize the driver implementation.

[nashif]

by Robert Beatty:

Leandro Pereira ,

Discussed today with Christopher Turner . We are going to add in driver-related stories to flesh this out better. These will help define the epic in terms of functionality and prioritization.