Web installer firmware update for 16M ESP32 results in 4M partition

ptr727 commented 2 years ago

What happened?

FYI, I discussed the problem with @srg74 on Discord, but wanted to document the issue here. Thanks again @srg74 for helping me get the correct partition layout. P.s., I still think there is something wrong with the TwilightLord ESP32 WiFi reception. I find it very unreliable, while QuinLED Uno and Quad ESP32's have no WiFi issues, but that is not the point of this report.

I bought a WLED Wemos Shield from @srg74 on Tindie. I removed the included TwilightLord-ESP32 16MB ESP32 and connected it to my Win PC, and flashed using the V2 Web Installer on Chrome. The resulting layout left 12MB unallocated, filesystem 12/983k.

I tried the ESPHome Flasher 1.4.0, using the 13b5 esp32_twilightlord_16mb.bin firmware file, and that failed with: [08:38:46].pio/libdeps/esp32_twilightlord_16mb/LittleFS_esp32/src/lfs.c:1076:error: Corrupted dir pair at {0x0, 0x1}

I wanted to flash using esptool 3.2, but I did not have the correct partition table binary, and they are not published.

I used ESP-IDF Docker to convert the partition table CSV to BIN:

docker run -it --rm --env TZ=America/Los_Angeles --volume c:/Users/piete/source/repos/esp:/project:rw espressif/idf /bin/bash
...
cd /opt/esp/idf/components/partition_table
python gen_esp32part.py /project/WLED_ESP32_16MB.csv /project/WLED_ESP32_16MB.bin

I tried to flash using esptool, but the default 0x8000 partition offset does not work, the bootloader is 64KB, so I need custom offsets, that I did not know how to figure out.

.\esptool.exe --port COM4 write_flash --erase-all 0x0 esp32_bootloader_v4.bin 0x8000 WLED_ESP32_16MB.bin 0x10000 esp32_twilightlord_16mb.bin
...
Detected overlap at address: 0x8000 for file: WLED_ESP32_16MB.bin

After lots of new learnings I ended up using VSCode and PlatformIO with @srg74 WLED_wemos_shield branch and uploaded the correct firmware, with the correct storage allocation, filesystem 12/4142kb.

To Reproduce Bug

Flash a 16MB ESP32 with the web flasher, partition is created as if it is a 4MB device.

Expected Behavior

To summarize: 1) The web flasher is not maximizing partitions based on device capabilities. Expectation is that the flasher id's the board and creates a partition utilizing all space. 2) The build is not publishing partition binaries. Expectation is that the build publishes all usable artifacts, including bootloader, partition, and app image. 3) The firmware updating instructions for pytools is lacking the correct parameters for flashing the bootloader, the partitions and the app image. Expectation is that the instructions include examples for various board types, i.e. 1MB, 4MB, 16MB partition layouts, with appropriate flash offsets.

Install Method

Binary from WLED.me

What version of WLED?

WLED 0.13.0-b5 (build 2111170)

Which microcontroller/board are you seeing the problem on?

ESP32

Relevant log/trace output

No response

Anything else?

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct

stale[bot] commented 2 years ago

Hey! This issue has been open for quite some time without any new comments now. It will be closed automatically in a week if no further activity occurs. Thank you for using WLED!

Aircoookie commented 2 years ago

Confirmed, the web installer is currently not (able to) detect flash size and will always flash the 4M binary. If https://github.com/esphome/esp-web-tools/issues/111 is implemented, the installer would be able to differentiate between different SPI flash sizes.

softhack007 commented 1 year ago

Closing, as this is not a WLED bug.

Please file a bug report for web installer in case the problem still exists.

https://github.com/Aircoookie/WLED-WebInstaller/issues

Aircoookie / WLED