Upgrade to ESP32 - Wemos S2/C3 mini and S3

[CheeseE]

Question Has anyone tried this or this board to upgrade the old ESP8266 to ESP32? Is it pin-to-pin compatible with the D1 mini? It seems to me but it would be good for someone to confirm, thanks.

[proddy]

The esp32-s2 is a different chip so the code needs to compiled. I've haven't tried it myself.

https://docs.platformio.org/en/stable/boards/espressif32/lolin_s2_mini.html

[bbqkees]

It's the easiest to just get a MH-ET LIVE D1 Mini with ESP32 chip. Pin compatible with the ESP8266 D1 Mini and it already has a board profile in the firmware.

[CheeseE]

Thank you, I have the MH-ET, just wanted to know if I have to cut the enclosure, as these mentioned boards are the same size as the D1 mini. Ordered them, and will try them out if it works.

[bbqkees]

The MH-ET is wider so you have to make a few adjustments to the housing. See the wiki: https://bbqkees-electronics.nl/wiki/gateway/upgrade-the-gateway-to-an-esp32.html

[MichaelDvP]

If you dont want to cut the housing, you can also cut the board and sidepanel:

[CheeseE]

What I'm trying to avoid is to cut the housing. But this is a good idea to cut the board itself, thank you :)

[MichaelDvP]

You also have to remove the plastic spacer from the board pins to make the usb fit without cutting the housing.

[ajvdw]

I did build for S2: there are 3 minor adjustments necessary NO UART2 moved to UART1 needs testing, NO BT skip include esp_bt.h and I modified the rename_fw script to generate a S2 specific bin file. Only problem: the settings are not stored in NVS. It's already mentioned as an issue in 3.4.2. If that part is fixed I can do some testing. I will create an PR if it works.

[CheeseE]

@ajvdw There's already a working version committed for Lolin C3 mini.

[ajvdw]

@ajvdw There's already a working version committed for Lolin C3 mini.

I followed the same path ;-)

I'd like to propose the bin files to follow the names in the PlatformIO Project environment, the rename_fw script would be something like, so no need to mention the flash size separately:

chip_target = env.get('PIOENV').upper()

print("app version: "+app_version)
print("chip_target: "+chip_target)
print("platform: "+platform)
# print("flash size: "+flash_size)

# convert . to _ so Windows doesn't complain
variant = "EMS-ESP-" + app_version.replace(".", "_") + "-" + chip_target 

[proddy]

That's a good recommendation. I had already change the file in the latest flash_optmization branch (which I'll merge into dev soon): https://github.com/emsesp/EMS-ESP32/blob/58a0ec9ccaefe5373044220507ce093096cde71b/scripts/rename_fw.py#L39 but your change is better. thanks

[MichaelDvP]

Today i've got a C3-Mini and a S2-Mini from lolin store. I'll check memory and issues the next days. S2 has less ram, but PSRAM. I'm curious how this works.

[proddy]

PSRAM

I'm also curious how to use it (there are a few ways described here). I'm also going to start looking at how to measure performance using some kind of profile. There must be something out there that can monitor. The alternative is to build a test into test.cpp which will stress test EMS-ESP by loading data, printing it, publishing it, and repeating for a few minutes, and then logging CPU and memory consumption. There are 2 cores in the ESP32 standard and I don't think we're even using the 2nd one fully.

[MichaelDvP]

C3 is running now with ems, dallas, analog, had some issues with gpio settings. Program size is larger, free ram a bit less then esp32, but working good. S2 seems to have uart1 fixed on gpio17/18, not pin compatible with D1 mini and no plug in replacement for gateway premium. psram is standard config as malloc, the dynamic json can use it and i think it is needed (without ems connection <50k free heap).

S2 without ems, dallas, analog connection, program size for single core is smaller:

[MichaelDvP]

S2 uart pins have to be configured before driver loading. Now it works as plugin in gateway premium. Psram is used without code change, no issues when calling api with large output. All working fast. Info with ems, dallas, analog connected:

[MichaelDvP]

@proddy To identify the platform in a string at central place i've modified the version.h to

#define EMSESP_APP_VERSION "3.5.0b5"

if defined(ARDUINO_LOLIN_C3_MINI)
#define EMSESP_PLATFORM "ESP32-C3";
#else
#if defined(ARDUINO_LOLIN_S2_MINI)
#define EMSESP_PLATFORM "ESP32-S2";
#else
#define EMSESP_PLATFORM "ESP32";
#endif
#endif

Is this ok, or should i add a platform.h with this defines?

[proddy]

Yes! Nice.

[MichaelDvP]

It's better to check the chip, not the specific board.

#if CONFIG_IDF_TARGET_ESP32C3
#define EMSESP_PLATFORM "ESP32-C3";
#elif CONFIG_IDF_TARGET_ESP32S2
#define EMSESP_PLATFORM "ESP32-S2";
#elif CONFIG_IDF_TARGET_ESP32
#define EMSESP_PLATFORM "ESP32";
#else
#error Target CONFIG_IDF_TARGET is not supported
#endif

The C3-Mini board have a rgb-LED (WS2812B), and can not be switchted by gpio, needs a fast protocoll with ns timing. Adafruit NeoLED lbrary can handle this, but flash is nearly full for ESP32C3. For now i skip LED.

[proddy]

I've ordered an S2 and C3 because I'm missing out on the fun too.

regarding the RGB, I have some old C code from another project years ago that handled the RGB LED on an ESP32, I'll see if I can reuse it.

[ajvdw]

I've done some testing with the S2, I can't change to GPIO 7 to 9 because of the validators. More GPIO's should be probably allowed. My board used GPIO18 and GP19 on the ESP32 on the S2 the GPIO pins 7 and 9 are on that location.... I defined my board profile but I couldn't get it to connect to the EMS bus. Probably because of incorrect initialization? I remember a remark about setting GPIO's before initializing the UART. But I can't find the matching commit with that fix.....

[proddy]

Those validators are easy to adjust for the different ESPs. I'm sure Michael can figure out the GPIOs init. I'm waiting for my S2's to arrive.

[MichaelDvP]

@ajvdw I will push a working dev soon to my repo and make a pr. Here is a working bin from my testings (with some extra debug logs, experimental values, etc.) EMS-ESP-LOLIN_S2_MINI-3_5_0b5.zip

[ajvdw]

@ajvdw I will push a working dev soon to my repo and make a pr. Here is a working bin from my testings (with some extra debug logs, experimental values, etc.) EMS-ESP-LOLIN_S2_MINI-3_5_0b5.zip

I've tested this version. It seems to work even with. the custom GPIO's set to 7 and 9.

[MichaelDvP]

The source is online now: https://github.com/MichaelDvP/EMS-ESP32/tree/dev_.

[proddy]

Got my S2 Mini and C3 Mini today so will join the fun

[MichaelDvP]

There is a new MH-ET formfactor S3 board LiLigo T7-S3, A bit expensive actually, but 16MB flash, 8MB PSRAM, dualcore xtensa LX7 (liligo writes RISC-V, but S3 is LX7), that should be enough for some years.

[bbqkees]

The ESP32-S3-WROOM-1-N16R8 is pretty expensive, with double the component price of the ESP32 16MB. But maybe a good one as a possible future successor.

[bbqkees]

I had a look at the datasheet. One benefit: it has USB built in. One bigger downside: It has ditched the convenient Ethernet MAC from the ESP32.

[MichaelDvP]

BTW: for testing something other i've connected the C3 buspowered parallel to my normal jackpowered ESP32. On startup, while AP is active, C3 gives some incomplete rx, but after AP switches off, complet stable with no rx errors. This is a real recommendation for people wanting upgrade from v2 to v3 but needs buspower (no servicejack, no power-jack).

[bbqkees]

I'll update the ESP8266 upgrade page on the wiki to add the new chips. Indeed if you are already on buspowered mode, being able to keep it is always nice.

[MichaelDvP]

Just a status message: I have some issues with S2 and it's small internal RAM. Arduinojson uses the psram, ok, but it seems wifi/tcp needs some free internal ram and if mqtt queue grow (due to a misconfiguration) wifi gets disconnected. Same if i set weblog to 100 buffer. We should check the free heap for these queues. (I've also tested with different settings of CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL and CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL, but did not find how to change this at complile time)

[proddy]

ugh, that's a pity. I think doing a free heap check (using heap_caps_get_largest_free_block()) is the best way forward, but not on every addition as that would create a lot of overhead. Perhaps calculate the available heap every 10 seconds in the main loop.

On my to-do list is also to see if using lighter-weight std libraries would ease up some memory. The std::string, map, array, queue eat up a lot of memory. In version 2 (ESP8266) I used https://github.com/muwerk/ustd for a while and it worked well.

[MichaelDvP]

I've tried with ESP.getMaxAllocHeap() < 10240 on every publish and reading the heap only in system.healthcheck (5sec). Working, S2 is stable and wifi is reachable. But on my system it starts dropping mqtt messages on ~150 queue-size. No problem for me, but ha-autodiscovery generates ~280 messages and ~130 of these are dropped. Better solution is to place all queues in psram, but don't know yet how to do this. BTW: web-log uses list and reducing the buffer size does not free mem, only indices are erased. Better to use deque.

[MichaelDvP]

Got the S3 https://www.lilygo.cc/products/t7-s3: Memory looks good (not connected to ems-bus).

[proddy]

I've added S3 to the list - this is our next gen ESP32 chip to focus on

[MichaelDvP]

From discussions:

@tp1de wrote:

@MichaelDvP the s3 version does not work stable. Dashboard reacting very slow and devices are found only after a long time. Could you explain how to flash new with your flasher? Maybe partioning is wrong by just uploading from VSC

Please reopen #605 and post logs there. Partitioning s ok, i think you get much garbage on the uart, is the gateway connected to the bus? Reopend..

[tp1de]

Please re-open this issue for Liligo T7 S3.

With the Liligo S3 and v3.6.0-dev.7 I do have the following findings:

• Built in one of the older ems-esp boards from @bbqkees and connecting to service jack, the board runs instable. I believe voltage is too low - if provided by service jack and board. With usb3 power supply the WebUI and WiFi is stable.

• I do have the old board and the new one connected in parallel to the ems-bus with different bus-id's. The Liligo S3 is not capable of reading all telegrams - only approx. 10% are recognized compared to the other one.

• Not all devices and only 20% of the entities are recognized. After some 10-20 minutes ems-esp is rebooting.

• EMS bus get's connected immediately. Quality is between 99 and 100% (due to the fact that 2 gateways are connected).

• All functions are working. Just telegram processing, identifying entities and devices are not working properly. The whole computing process seems to run with very very low speed.

• EMS-ESP Version v3.6.0-dev.7 Device (Platform / SDK)...........ESP32-S3 / v4.4.2 System Uptime............................000+00:02:39.115 CPU Frequency ..........................240 MHz... Heap (Free / Max Alloc) .........188 KB / 179 KB PSRAM (Size / Free) .................8.189 KB / 8.162 KB Flash Chip (Size / Speed) ..... 16.384 KB / 80 MHz Application (Used / Free) ......1.868 KB / 6.260 KB File System (Used / Free) .......28 KB / 36 KB

[MichaelDvP]

Could you send a full log. Have you set different Bus-ids to the two ems-esp? unplug the voltage regulator for usb-power? I also see you are on SDK 4.4.2 (platform 5.2) , please add to your pio_local.ini

[env:lolin_s3]
platform = espressif32
build_flags = ${common.build_flags} -DEMSESP_DEBUG

then it's easier to compare to mine. Make the log after new compile to have the debug messages. For compare, ET-MH32 on service jack, bus-id 0x0B, Liligo S3 on bus, usb powered, bus-id 0x12:

EMS-ESP Version v3.6.0-dev.7
Platform (Platform / SDK) ESP32-S3 / v4.4.4
System Betriebszeit 000+01:14:14.805
CPU Frequenz 240 MHz
freier RAM Speicher (Gesamt / max. Block) 192 KB / 167 KB
PSRAM (Größe / Frei) 8.189 KB / 8.169 KB
Flash Speicher (Größe / Geschwindigkeit) 16.384 KB / 80 MHz
Programm (Genutzt / Frei) 1.915 KB / 6.213 KB
Dateisystem (Genutzt / Frei) 28 KB / 36 KB

EMS-Telegramme empfangen (Rx) | 8.805 | 2 | 100%
EMS-Telegramme gelesen (Tx) | 1.162 | 2 | 100%
EMS-Telegramme geschrieben (Tx) | 0 | 0 |  
Temperatursensoren gelesen | 0 | 0 |  
Analogsensoren gelesen | 2.494 | 0 | 100%
MQTT-Nachrichten gesendet | 3.566 | 0 | 100%
API-Aufrufe | 0 | 0 |  

[tp1de]

@MichaelDvP The Lilygo S3 does not work properly. Compiling in debug modes makes the gateway not responding anymore. Without debug flag I could access but response is incredible slow. E.g. looking at system log takes more then 20 seconds to display each line .... I feel that I should flash with your flasher. Could you please explain how to do it.

[MichaelDvP]

I think you have a configuration error with the two ems-esp, same bus-id, same mqtt-settings, same hostname, or something like that. But without log and system info it's just a look in a crystall ball.

[tp1de]

No different bus-id, s3 without mqtt, different hostname and ip-adresses. Sorry I can not display system log since it is basically not displayed and/or system reboots in the meantime. Even with just gateway (s3) it does not work. No chance to flash with your flasher?

[MichaelDvP]

I have tried the official build, it crashes every few seconds. Seems i have made a fix in my testbuild, have to check what is was. (Edit: removed testbuild to avoid confusion)

[MichaelDvP]

Seems to be a stack overflow in UART task, change line 92 of emsuart_esp32.cpp to xTaskCreate(uart_event_task, "uart_event_task", 2560, NULL, configMAX_PRIORITIES - 1, NULL); (change stack size from 2k to 2,5k) (my build uses 2,2k)

[tp1de]

@MichaelDvP yes uart stack size was causing the problems. Now S3 works, I just have to find out why with the S3 version I do have 2 thermostat entities less.

[tp1de]

Under S3 the 2 following thermostat entities are missing: hc1/hc2.haclimate. Anyhow I do have no glue what these entities are used for. Same code for S3 (16MB) and ESP32 (4MB) with new partioning.

[MichaelDvP]

This is a dummy entity to generate the homeassistant climate card depending on thermostat configuration. It should show up in settings customizations, but only has a value if HA is activated.

[tp1de]

This is a dummy entity to generate the homeassistant climate card depending on thermostat configuration. It should show up in settings customizations, but only has a value if HA is activated.

Understood. Everything seems to work now. Just a final question:

while using the S3 I need to compile always by myself - isn't it? (I can't use the bin-files (4MB) provided by @proddy)

[bbqkees]

The current bin file on the repository is specific to the ESP32 architecture, the ESP32-S3 is a different architecture. Maybe it's an idea to start offering a bin file for the S3 as well so more people can test it easily.

[MichaelDvP]

The esp32, s2, c3, s3 are different chip types and needs different binary files. The platformio.ini sets the right chiptype and partition size. For now you need to compile yourself, maybe later there will be different binaries for download. The web upload checks file header and you can not upload a wrong binary and get http:503 service unavailable for wrong binaries.

Set in your pio_local.ini

[platformio]
 default_envs = esp32_16M, lolin_s3

to build both binaries if you hit build button.

[proddy]

I've been running on S3 for a while now, very stable. @MichaelDvP do you think we can close this?