Add support for esp32s3 and custom version with different SPIRAM CS Pin

[TGiles1998]

Operating system macOS Mojave

Python version 3.10

What Chip ESP32 S2 16MB

Uploaded latest 16MB. Spiram Micropython

esptool.py --port /dev/cu.SLAB_USBtoUART -b 460800 --before default_reset --after hard_reset --chip esp32 write_flash --flash_mode dio --flash_size detect --flash_freq 40m 0x1000 bootloader/bootloader.bin 0x8000 partition_table/partition-table.bin 0x10000 micropython.bin

ampy -p /dev/cu.SLAB_USBtoUART put main.py ampy -p /dev/cu.SLAB_USBtoUART put i2s_dump.py ampy -p /dev/cu.SLAB_USBtoUART put micro_speech.py ampy -p /dev/cu.SLAB_USBtoUART put model.tflite

Result returned:

ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0030,len:4344 load:0x40078000,len:13816 load:0x40080400,len:3340 entry 0x40080618 Traceback (most recent call last): File "main.py", line 14, in ImportError: no module named 'audio_frontend' MicroPython 36f7ca0 on 2021-11-09; ESP32 module (microlite-spiram-16m) with ESP32 Type "help()" for more information.

Is the audio frontedn part of this firmware or do I need to import this seperately and if so where might I find the file for audio_frontend

Many thanks,

Thomas Giles

[mocleiri]

At the moment the only place the audio frontend module is working in is for esp32 built from the main branch. like: https://github.com/mocleiri/tensorflow-micropython-examples/actions/runs/1176418468

I'm working toward supporting circuit python and I need to have everything in one module for that so in the future it will be a class within the microlite module.

The current builds for stm32 (broken), esp32 (working) and rp2 (working) are using the newer approach of building tflm within the micropython build and I haven't finished setting up the audio frontend yet. See #36.

[TGiles1998]

Thank you so much! I'm currently trying to build this for a ESP32-S3.

[mocleiri]

@TGiles1998 can you point me to the board config in micropython for your board?

I can try to add a microlite firmware build for it as part of the GitHub actions CI.

Would it be GENERIC_S3_SPIRAM? https://github.com/micropython/micropython/tree/master/ports/esp32/boards

[TGiles1998]

@mocleiri Ive currently built a working standard Micropython for ESP32-S3 with 8MB flash and 8MB PSRAM. Using SPIRAM CS as GPIO 20.

I think these are the relevant config options. Once again thank you so much!

ESP32S3-Specific

CONFIG_ESP32S3_DEFAULT_CPU_FREQ_80 is not set CONFIG_ESP32S3_DEFAULT_CPU_FREQ_160=y CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240 is not set CONFIG_ESP32S3_DEFAULT_CPU_FREQ_MHZ=160

Cache config

CONFIG_ESP32S3_INSTRUCTION_CACHE_16KB=y CONFIG_ESP32S3_INSTRUCTION_CACHE_32KB is not set CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE=0x4000 CONFIG_ESP32S3_INSTRUCTION_CACHE_4WAYS is not set CONFIG_ESP32S3_INSTRUCTION_CACHE_8WAYS=y CONFIG_ESP32S3_ICACHE_ASSOCIATED_WAYS=8 CONFIG_ESP32S3_INSTRUCTION_CACHE_LINE_16B is not set CONFIG_ESP32S3_INSTRUCTION_CACHE_LINE_32B=y CONFIG_ESP32S3_INSTRUCTION_CACHE_LINE_SIZE=32 CONFIG_ESP32S3_INSTRUCTION_CACHE_WRAP is not set CONFIG_ESP32S3_DATA_CACHE_16KB is not set CONFIG_ESP32S3_DATA_CACHE_32KB=y CONFIG_ESP32S3_DATA_CACHE_64KB is not set CONFIG_ESP32S3_DATA_CACHE_SIZE=0x8000 CONFIG_ESP32S3_DATA_CACHE_4WAYS is not set CONFIG_ESP32S3_DATA_CACHE_8WAYS=y CONFIG_ESP32S3_DCACHE_ASSOCIATED_WAYS=8 CONFIG_ESP32S3_DATA_CACHE_LINE_16B is not set CONFIG_ESP32S3_DATA_CACHE_LINE_32B=y CONFIG_ESP32S3_DATA_CACHE_LINE_64B is not set CONFIG_ESP32S3_DATA_CACHE_LINE_SIZE=32 CONFIG_ESP32S3_DATA_CACHE_WRAP is not set end of Cache config

CONFIG_ESP32S3_SPIRAM_SUPPORT=y

SPI RAM config

CONFIG_SPIRAM_MODE_QUAD=y CONFIG_SPIRAM_MODE_OCT is not set CONFIG_SPIRAM_TYPE_AUTO is not set CONFIG_SPIRAM_TYPE_ESPPSRAM16 is not set CONFIG_SPIRAM_TYPE_ESPPSRAM32 is not set CONFIG_SPIRAM_TYPE_ESPPSRAM64=y CONFIG_SPIRAM_SIZE=8388608

PSRAM Clock and CS IO for ESP32S3

CONFIG_DEFAULT_PSRAM_CLK_IO=30 CONFIG_DEFAULT_PSRAM_CS_IO=20 end of PSRAM Clock and CS IO for ESP32S3

CONFIG_SPIRAM_FETCH_INSTRUCTIONS is not set CONFIG_SPIRAM_RODATA is not set CONFIG_SPIRAM_SPEED_120M is not set CONFIG_SPIRAM_SPEED_80M=y CONFIG_SPIRAM_SPEED_40M is not set CONFIG_SPIRAM=y CONFIG_SPIRAM_BOOT_INIT=y CONFIG_SPIRAM_IGNORE_NOTFOUND=y CONFIG_SPIRAM_USE_MEMMAP is not set CONFIG_SPIRAM_USE_CAPS_ALLOC=y CONFIG_SPIRAM_USE_MALLOC is not set CONFIG_SPIRAM_MEMTEST is not set CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP is not set end of SPI RAM config

CONFIG_ESP32S3_TRAX is not set CONFIG_ESP32S3_TRACEMEM_RESERVE_DRAM=0x0 CONFIG_ESP32S3_ULP_COPROC_ENABLED is not set CONFIG_ESP32S3_ULP_COPROC_RESERVE_MEM=0 CONFIG_ESP32S3_DEBUG_OCDAWARE=y CONFIG_ESP32S3_DEBUG_STUBS_ENABLE is not set CONFIG_ESP32S3_BROWNOUT_DET=y CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_7=y CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_6 is not set CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_5 is not set CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_4 is not set CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_3 is not set CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_2 is not set CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_1 is not set CONFIG_ESP32S3_BROWNOUT_DET_LVL=7 CONFIG_ESP32S3_TIME_SYSCALL_USE_RTC_FRC1=y CONFIG_ESP32S3_TIME_SYSCALL_USE_RTC is not set CONFIG_ESP32S3_TIME_SYSCALL_USE_FRC1 is not set CONFIG_ESP32S3_TIME_SYSCALL_USE_NONE is not set CONFIG_ESP32S3_RTC_CLK_SRC_INT_RC=y CONFIG_ESP32S3_RTC_CLK_SRC_EXT_CRYS is not set CONFIG_ESP32S3_RTC_CLK_SRC_EXT_OSC is not set CONFIG_ESP32S3_RTC_CLK_SRC_INT_8MD256 is not set CONFIG_ESP32S3_RTC_CLK_CAL_CYCLES=1024 CONFIG_ESP32S3_DEEP_SLEEP_WAKEUP_DELAY=2000 CONFIG_ESP32S3_NO_BLOBS is not set CONFIG_ESP32S3_RTCDATA_IN_FAST_MEM is not set CONFIG_ESP32S3_USE_FIXED_STATIC_RAM_SIZE is not set end of ESP32S3-Specific

[mocleiri]

@TGiles1998

I have a feature branch working to integrate s3 support.

I've setup the custom board directories from the micropython ports/esp32/boards/GENERIC_S3 and ports/esp32/boards/GENERIC_S3_SPIRAM boards but I did it at the main branch level first.

I don't think it will work there because I don't know how to configure the tensorflow make file to build tensorflow for the esp32s3.

I will switch gears and instead add these board configs on the latest code which builds tlfm as part of the microlite build within the micropython build which does know how to build for esp32s3.

The downside for you is that the audio frontend isn't integrated there yet.

Is the SPIRAM CS as GPIO 20 something physical on your board or a software configuration choice?

The upstream base configs in micropython use GPIO 26: https://github.com/micropython/micropython/blob/master/ports/esp32/boards/sdkconfig.spiram_sx#L7

I am fine with adding exactly what you have its just I want to know if yours is the base board or a custom board config.

Also at least with regular esp32 running at 240Mhz was required for micro_speech for the system to keep up with the audio sampling and spectrogramming and ml inferencing.

Many of the other config settings don't seem to be in the micropython configs which I think means they are using the defaults.

[TGiles1998]

@mocleiri Hi the base board uses GPIO26, it would be best to use that so that more people can then use the software once Espressif brings out the hardware, I believe they have a S3-wrover type module in the works atm. I just tried to create a momentary solution with making my own hardware. Setting up with the GENERIC_S3_SPIRAM would be absolutely excellent! I also managed to find a work around for not having the audio-frontend so don't worry! many thanks once again,

a picture of my custom hardware is below (which required some delicate soldering).

[mocleiri]

@TGiles1998 I've successfully migrated the audio_frontend to be within the microlite module. I'm close to being able to build the esp32s3 firmware, the ci is running and failing but I have a good idea where and why.

I think within a day or two I'll have the standard s3 firmwares building. Then I will add one for your board with the different options you need

[TGiles1998]

@mocleiri Thats amazing! Thank you so much!

[mocleiri]

@TGiles1998 the build succeeded can you test the esp32s3 CS gpio 20 board here: https://github.com/mocleiri/tensorflow-micropython-examples/actions/runs/1590018014

If you scroll down you should be able to download the zip file and flash it.

This is base with one change to set the spi CS pin to gpio 20: https://github.com/mocleiri/tensorflow-micropython-examples/blob/4b6f4113aa12e5be7d978257080320aa01742e3e/boards/esp32/MICROLITE_S3_SPIRAM_CS_GPIO_20/sdkconfig.board#L15

If this works I can add any of the sdk properties you specified to further customize.

[TGiles1998]

@mocleiri Thank you so much for your reply and the time put into integrating this solution for ESP32S3!

I have flashed to the board with the custom files and unfortunately it produced the error below:

Saved PC:0x403bb352 SPIWP:0xee mode:DIO, clock div:1 load:0x3fcd0108,len:0xf60 load:0x403b6000,len:0x978 load:0x403ba000,len:0x2c80 entry 0x403b616c W (24) bootloader_random: RNG for ESP32-S3 not currently supported E (24) flash_parts: partition 3 invalid magic number 0x3020 E (26) boot: Failed to verify partition table E (31) boot: load partition table error! ESP-ROM:esp32s3-20210327 Build:Mar 27 2021 rst:0x3 (RTC_SW_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)

I have had this error before with the standard micro-python build when trying to communicate to it, to verify it works using a REPL such as Screen or PyMakr. https://github.com/micropython/micropython/issues/8053#issuecomment-983787470_

The solution was to remove boards/sdkconfig.usb from: micropython/ports/esp32/boards/GENERIC_S3_SPIRAM/mpconfigboard.cmake

This resets communication to the board back to the standard UART and it works perfectly.

Would it be possible to do this for the files you have created?

Once again I am very grateful for all your continued help!

[mocleiri]

I'll make the change later today.

As far as I'm concerned this custom config can be whatever options you'd like.

Once it works feel free to request other options. Or file a pull request and I'll apply exactly the options you are using locally.

[mocleiri]

@TGiles1998

I removed the sdkconfig.usb and the new firmware is available here: https://github.com/mocleiri/tensorflow-micropython-examples/actions/runs/1595108062

I think if it works lets close this issue and then open up other ones for any other adjustments you'd like to make.

[TGiles1998]

@mocleiri

Hi yes that firmware seems to be working, I have been able to talk to the external RAM and use the micropython functions. Thank you so much!

However, I then tried to run the Micro-Speech example and unfortunately this error occurred:

Connecting to /dev/tty.SLAB_USBtoUART... ESP-ROM:esp32s3-20210327 Build:Mar 27 2021 rst:0x10 (RTCWDT_RTC_RST),boot:0x8 (SPI_FAST_FLASH_BOOT) SPIWP:0xee mode:DIO, clock div:1 load:0x3fcd0108,len:0xf60 load:0x403b6000,len:0x978 load:0x403ba000,len:0x2c80 entry 0x403b616c W (25) bootloader_random: RNG for ESP32-S3 not currently supported W (313) bootloader_random: RNG for ESP32-S3 not currently supported Traceback (most recent call last): File "main.py", line 14, in ImportError: no module named 'audio_frontend' MicroPython 59e6194 on 2021-12-18; ESP32S3 module (microlite-spiram) with ESP32S3 Type "help()" for more information.

However, as you previously suggested I will close this issue as the microlite is working for external RAM on GPIO 20 CS. And will open a new issue.

Thanks again!