search

AchimPieters / esp32-homekit

esp32-homekit
https://www.studiopieters.nl
MIT License
7 stars 2 forks source link

Add support for staging component name mywolfssl #1

Closed gojimmypi closed 5 months ago

gojimmypi commented 5 months ago

The staging instance component name of wolfSSL is called mywolfssl instead of wolfssl to ensure there's no ambiguity between release and test/ preview code.

https://components-staging.espressif.com/components/gojimmypi/mywolfssl

AchimPieters commented 5 months ago

@gojimmypi 

FAILED: esp-idf/esptool_py/CMakeFiles/app_check_size /project/examples/led/build/esp-idf/esptool_py/CMakeFiles/app_check_size 
cd /project/examples/led/build/esp-idf/esptool_py && /opt/esp/python_env/idf5.4_py3.10_env/bin/python /opt/esp/idf/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /project/examples/led/build/partition_table/partition-table.bin /project/examples/led/build/main.bin
Error: app partition is too small for binary main.bin size 0x1038b0:
  - Part 'factory' 0/0 @ 0x10000 size 0x100000 (overflow 0x38b0)
ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in the /project/examples/led/build/log/idf_py_stderr_output_1430 and /project/examples/led/build/log/idf_py_stdout_output_1430
gojimmypi commented 5 months ago

Given https://github.com/AchimPieters/esp32-homekit-demo/pull/12#issuecomment-2133749247 regarding resolved flash footprint size, @AchimPieters do you think it would still be a good idea to turn off hardware acceleration for all 1MB flash to conserve flash size?

We could look for #define CONFIG_ESPTOOLPY_FLASHSIZE_1MB 1 in the sdkconfig.h; I'd add a Kconfig "Disable hardware acceleration for 1MB devices" option as well, and set this only as needed, such as when the Apple HomeKit is enabled, something like this in user_settings'h:

#if defined(WOLFSSL_APPLE_HOMEKIT) || defined(CONFIG_WOLFSSL_APPLE_HOMEKIT)
    /* SRP is known to need 8K; slow on some devices */
    #undef  FP_MAX_BITS
    #define FP_MAX_BITS (8192 * 2)
    #define WOLFCRYPT_HAVE_SRP
    #define HAVE_CHACHA
    #define HAVE_POLY1305
    #define WOLFSSL_BASE64_ENCODE
    #define HAVE_HKDF
    #define WOLFSSL_SHA512
    #if defined(CONFIG_ESPTOOLPY_FLASHSIZE_1MB) && defined(CONFIG_WOLFSSL_NO_1MB_ESP32_CRYPT) 
        #define NO_ESP32_CRYPT
    #endif
 #endif
AchimPieters commented 5 months ago

@gojimmypi OR we just say you need an ESP32 with at least 4MB, and use the optimized settings, for the best result, with the lowest impact on performance and stability, and as lean as we can get it?

gojimmypi commented 5 months ago

I still like your idea of "it just works on every device, everywhere".

Was your resolution a revised partition file, and you didn't actually have a 1MB ESP32? Curious I did not see that error when building from a fresh project. I assumed your build actually found a 1MB device and used that limit, eh?

btw - that capture of an ESP32 Partition Calculator is pretty cool. Is it this one: https://esp32.jgarrettcorbin.com/ ?

Let me know where we are with things: what still needs attention? If all is looking good with the staging component I will update the release preview.

AchimPieters commented 5 months ago

@gojimmypi

Yes, my resolution was a revised partition file. I didn't actually have a 1MB ESP32; I just configured the partition file to reflect that size.

Regarding the error you mentioned, it didn't occur because the partition file was manually set that way, not because the build found a 1MB device.

And yes, the ESP32 Partition Calculator you saw is the one available at https://esp32.jgarrettcorbin.com/.

As for the current status, the tests I've conducted so far are working fine. If everything looks good with the staging component, feel free to update the release preview.

gojimmypi commented 5 months ago

Hi @AchimPieters

That's interesting about the partition file... for several reasons.

What version of the ESP-IDF are you using? I'm using v5.2.

I've been doing regression tests to make sure all the devices work with the latest preview version of wolfSSL. My ESP32-C2 has only 2MB of flash. I am able to program it successfully, yet the device will not boot. I worked on this for some time before considering the updated partition file, as there's no warning or other indication of memory size problem. It just fails with this message at boot time:

invalid header: 0x852▒ESP-ROM:esp8684-api2-20220127
Build:Jan 27 2022
rst:0x7 (TG0WDT_SYS_RST),boot:0xc (SPI_FAST_FLASH_BOOT)
Saved PC:0x40059706
invalid header: 0x85228b2a
invalid header: 0x85228b2a
invalid header: 0x85228b2a
invalid header: 0x85228b2a
invalid header: 0x85228b2a
[...repeats many times and then reboots...]

Here's the log header with details that are displayed with the wolfssl_test app instead loaded on that ESP32-C2:

SP-ROM:esp8684-api2-20220127
Build:Jan 27 2022
rst:0x1 (POWERON),boot:0xc (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5c80,len:0x16f0
load:0x403acb70,len:0xbb0
load:0x403aeb70,len:0x27e0
entry 0x403acb7a
I (21) boot: ESP-IDF v5.2-dev-3903-g66992aca7a-dirty 2nd stage bootloader
I (21) boot: compile time May 28 2024 22:06:37
I (22) boot: chip revision: v1.0
I (26) boot.esp32c2: MMU Page Size  : 32K
I (31) boot.esp32c2: SPI Speed      : 60MHz
I (36) boot.esp32c2: SPI Mode       : DIO
I (40) boot.esp32c2: SPI Flash Size : 2MB
I (45) boot: Enabling RNG early entropy source...
I (50) boot: Partition Table:
I (54) boot: ## Label            Usage          Type ST Offset   Length
I (61) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (69) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (76) boot:  2 factory          factory app      00 00 00010000 00177000
I (84) boot: End of partition table
I (88) esp_image: segment 0: paddr=00010020 vaddr=3c0b0020 size=2e1e0h (188896)                                              map
I (139) esp_image: segment 1: paddr=0003e208 vaddr=3fcaa500 size=0175ch (  5980)                                              load
I (140) esp_image: segment 2: paddr=0003f96c vaddr=40380000 size=006ach (  1708)                                              load
I (145) esp_image: segment 3: paddr=00040020 vaddr=42000020 size=ad3a8h (709544)                                              map
I (312) esp_image: segment 4: paddr=000ed3d0 vaddr=403806ac size=09e4ch ( 40524)                                              load
I (325) boot: Loaded app from partition at offset 0x10000
I (326) boot: Disabling RNG early entropy source...
I (337) cpu_start: Unicore app
I (346) cpu_start: Pro cpu start user code
I (346) cpu_start: cpu freq: 120000000 Hz
I (346) cpu_start: Application information:
I (349) cpu_start: Project name:     wolfssl_test
I (354) cpu_start: App version:      v5.7.0-stable-2294-ga60d2986c-d
I (361) cpu_start: Compile time:     May 28 2024 22:06:15
I (367) cpu_start: ELF file SHA256:  54f57b3d5...
I (373) cpu_start: ESP-IDF:          v5.2-dev-3903-g66992aca7a-dirty
I (380) cpu_start: Min chip rev:     v1.0
I (384) cpu_start: Max chip rev:     v1.99
I (389) cpu_start: Chip rev:         v1.0
I (394) heap_init: Initializing. RAM available for dynamic allocation:
I (401) heap_init: At 3FCACF90 len 0002FBE0 (190 KiB): RAM
I (407) heap_init: At 3FCDCB70 len 0000294C (10 KiB): RAM
I (414) spi_flash: detected chip: generic
I (418) spi_flash: flash io: dio
I (422) sleep: Configure to isolate all GPIO pins in sleep state
I (429) sleep: Enable automatic switching of GPIO sleep configuration

As this is unrelated to wolfSSL, I'll complete a bit more testing and then publish a release preview soon.

Thanks for all your help. This exercise has been educational for us both!

Cheers

AchimPieters commented 5 months ago

@gojimmypi I remembered, why I set the max to 1MB, because I wanted to use LCM in combination with this module: https://nl.aliexpress.com/item/1005004708883228.html?spm=a2g0o.order_list.order_list_main.192.c5c379d2ctD3Rj&gatewayAdapt=glo2nld It has 2 MB and I need two times an OTA partition for LCM, so it needs to be smaller than <1MB. These modules are commonly used is Smart devices, like a smart plug, smart switch or smart sensor...

gojimmypi commented 5 months ago

I set the max to 1MB, because I wanted to use LCM

ah yes, I see. Well, I have several other devices that are also failing with the partition changes. Not only is my ESP32-C2 a 2MB device, but also ESP32-C3, ESP32-S2, and ESP32-S3.

So perhaps we're back to fitting into a 1MB partition? What do you see as the best method? There might be other flash-saving options to consider as well.

gojimmypi commented 5 months ago

I should note that although the staging components are working from command-line (e.g. WSL), I've not been able to get that working in VisualGDB. See:

https://sysprogs.com/w/forums/topic/espressif-esp32-staging-components-preview-fails-to-find-component/

AchimPieters commented 5 months ago

@gojimmypi I'm going to look into this one, this weekend, but don't have the time I would like. I also have found some optimizations options.

AchimPieters commented 5 months ago

@gojimmypi

To optimize speed, minimize binary size, and minimize RAM usage for an ESP32 project using the ESP-IDF, I configured the SDK using the sdkconfig.defaults file. Based on the guidelines from the Espressif documentation, here are the configurations I added:

Enable Compiler Optimization: CONFIG_COMPILER_OPTIMIZATION_SIZE=y This sets the compiler to optimize for size, reducing binary size.

Enable LTO (Link Time Optimization): CONFIG_COMPILER_OPTIMIZATION_LTO=y LTO can significantly reduce binary size and sometimes improve execution speed.

Binary Size Reduction Reduce Log Level: CONFIG_LOG_DEFAULT_LEVEL_ERROR=y This reduces the amount of log output, minimizing binary size and runtime overhead.

Disable Unused Components:

CONFIG_BT_ENABLED=n
CONFIG_BLE_ENABLED=n
CONFIG_ESP_CONSOLE_UART_DEFAULT=n

Disable Bluetooth, BLE, and default UART console if not needed to save space.

Exclude Unnecessary Features:

CONFIG_ESP32_PHY_CALIBRATION_AND_DATA_STORAGE=n
CONFIG_ESP32_PANIC_PRINT_HALT=n
CONFIG_ESP32_REDUCE_PHY_TX_POWER=y

RAM Usage Minimization

Optimize Task Stack Sizes:

CONFIG_FREERTOS_IDLE_TASK_STACKSIZE=512
CONFIG_FREERTOS_ISR_STACKSIZE=1536

Adjust stack sizes for the idle and ISR tasks.

Optimize Heap Configuration:

CONFIG_HEAP_POISONING_DISABLED=y
CONFIG_HEAP_TRACING_OFF=y

Reduce TCP/IP Stack Size:

CONFIG_TCPIP_RECVMBOX_SIZE=32
CONFIG_TCPIP_SENDMBOX_SIZE=32
CONFIG_TCP_MSS=536

Here’s the sdkconfig.defaults file with the configurations mentioned:

# General Optimization
# General Optimization
CONFIG_COMPILER_OPTIMIZATION_SIZE=y
CONFIG_COMPILER_OPTIMIZATION_LTO=y

# Binary Size Reduction
CONFIG_LOG_DEFAULT_LEVEL_ERROR=y
CONFIG_BT_ENABLED=n
CONFIG_BLE_ENABLED=n
CONFIG_ESP_CONSOLE_UART_DEFAULT=n
CONFIG_ESP32_PHY_CALIBRATION_AND_DATA_STORAGE=n
CONFIG_ESP32_PANIC_PRINT_HALT=n
CONFIG_ESP32_REDUCE_PHY_TX_POWER=y

# RAM Usage Minimization
# CONFIG_FREERTOS_IDLE_TASK_STACKSIZE=512
# CONFIG_FREERTOS_ISR_STACKSIZE=1536
CONFIG_HEAP_POISONING_DISABLED=y
CONFIG_HEAP_TRACING_OFF=y
CONFIG_TCPIP_RECVMBOX_SIZE=32
CONFIG_TCPIP_SENDMBOX_SIZE=32
CONFIG_TCP_MSS=536

main.bin binary size 0xe4de0 bytes. Smallest app partition is 0x100000 bytes. 0x1b220 bytes (11%) free

You can further customize these settings based on your specific application (WOLFSSL) requirements. Make sure to test the performance and stability of your application after applying these optimizations.

PS. if I want to use LCM on a 2MB ESP32C2 the maximum is 800K!