adafruit / tinyuf2

UF2 bootloader based on TinyUSB for embedded devices such as ESP32S2, STM32F4 and iMX RT10xx
MIT License
314 stars 172 forks source link

ESP32 Build is broken, Does not enumerate #268

Closed SukuWc closed 1 year ago

SukuWc commented 1 year ago

Operating System

Linux

INFO_UF2.TXT

TinyUF2 Bootloader 0.11.0-13-g9796a54 - tinyusb (0.12.0-203-ga4cfd1c69) Model: Unexpected Maker TinyS3 Board-ID: ESP32S3-TinyS3-01 Date: Nov 8 2022

What happened ?

ESP32 build worked fine until mid november, since then it is broken, it does not enumerate. Maybe it is cause by updated docker image for esp-idf. The image used by the workflow is repressive/idf:release-v4.4 which may have been updated at that time... last updated 5 days ago!

How to reproduce ?

I am testing on unexpectedmaker tinys3 board! Here are the last working and first non-working builds that I found!

(Nov 9) unexpectedmaker_tiny3 boots fine https://github.com/adafruit/tinyuf2/actions/runs/3424066037

(Nov 15) couple of commits later it does not enumerate https://github.com/adafruit/tinyuf2/actions/runs/3466891115

Can anyone try to reproduce the issue? What could be wrong?

Debug Log

No response

Screenshots

No response

SukuWc commented 1 year ago

I just tested it with the unexpectedmaker pros3 board as well, same thing happens. The builds after Nov15 do not enumerate, the LED stays RED!

prplz commented 1 year ago

Confirmed on tinys3, it is boot looping


Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd0108,len:0x1644
load:0x403b6000,len:0xe7c
load:0x403ba000,len:0x3248
entry 0x403b6334
␛[0;32mI (25) boot: ESP-IDF v4.4.3-145-g76caa64286 2nd stage bootloader␛[0m
␛[0;32mI (25) boot: compile time 02:29:08␛[0m
␛[0;32mI (25) boot: chip revision: 0␛[0m
␛[0;32mI (28) qio_mode: Enabling default flash chip QIO␛[0m
␛[0;32mI (33) boot.esp32s3: Boot SPI Speed : 80MHz␛[0m
␛[0;32mI (38) boot.esp32s3: SPI Mode       : QIO␛[0m
␛[0;32mI (43) boot.esp32s3: SPI Flash Size : 8MB␛[0m
␛[0;32mI (47) boot: Enabling RNG early entropy source...␛[0m
␛[0;32mI (53) boot: Partition Table:␛[0m
␛[0;32mI (56) boot: ## Label            Usage          Type ST Offset   Length␛[0m
␛[0;32mI (64) boot:  0 nvs              WiFi data        01 02 00009000 00005000␛[0m
␛[0;32mI (71) boot:  1 otadata          OTA data         01 00 0000e000 00002000␛[0m
␛[0;32mI (79) boot:  2 ota_0            OTA app          00 10 00010000 00200000␛[0m
␛[0;32mI (86) boot:  3 ota_1            OTA app          00 11 00210000 00200000␛[0m
␛[0;32mI (93) boot:  4 uf2              factory app      00 00 00410000 00040000␛[0m
␛[0;32mI (101) boot:  5 ffat             Unknown data     01 81 00450000 003b0000␛[0m
␛[0;32mI (109) boot: End of partition table␛[0m
␛[0;32mI (113) boot: Defaulting to factory image␛[0m
␛[0;32mI (117) esp_image: segment 0: paddr=00410020 vaddr=3c020020 size=044c4h ( 17604) map␛[0m
␛[0;32mI (129) esp_image: segment 1: paddr=004144ec vaddr=3fc90620 size=01dcch (  7628) load␛[0m
␛[0;32mI (136) esp_image: segment 2: paddr=004162c0 vaddr=40374000 size=09d58h ( 40280) load␛[0m
␛[0;32mI (151) esp_image: segment 3: paddr=00420020 vaddr=42000020 size=13d00h ( 81152) map␛[0m
␛[0;32mI (164) esp_image: segment 4: paddr=00433d28 vaddr=4037dd58 size=028bch ( 10428) load␛[0m
␛[0;32mI (176) boot: Loaded app from partition at offset 0x410000␛[0m
␛[0;32mI (176) boot: Disabling RNG early entropy source...␛[0m
␛[0;32mI (188) cpu_start: Pro cpu up.␛[0m
␛[0;32mI (188) cpu_start: Starting app cpu, entry point is 0x40374d6c␛[0m
␛[0;32mI (0) cpu_start: App cpu up.␛[0m
␛[0;32mI (202) cpu_start: Pro cpu start user code␛[0m
␛[0;32mI (202) cpu_start: cpu freq: 160000000␛[0m
␛[0;32mI (202) cpu_start: Application information:␛[0m
␛[0;32mI (205) cpu_start: Project name:     tinyuf2␛[0m
␛[0;32mI (210) cpu_start: App version:      0.11.0-15-g1777d44␛[0m
␛[0;32mI (215) cpu_start: Compile time:     Nov 15 2022 02:29:04␛[0m
␛[0;32mI (222) cpu_start: ELF file SHA256:  30b5c0ed93c53172...␛[0m
␛[0;32mI (228) cpu_start: ESP-IDF:          v4.4.3-145-g76caa64286␛[0m
␛[0;32mI (234) heap_init: Initializing. RAM available for dynamic allocation:␛[0m
␛[0;32mI (241) heap_init: At 3FCA5AC8 len 00043C48 (271 KiB): D/IRAM␛[0m
␛[0;32mI (252) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM␛[0m
␛[0;32mI (254) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM␛[0m
␛[0;32mI (260) heap_init: At 600FE000 len 00002000 (8 KiB): RTCRAM␛[0m
␛[0;32mI (267) spi_flash: detected chip: gd␛[0m
␛[0;32mI (271) spi_flash: flash io: qio␛[0m
␛[0;32mI (275) sleep: Configure to isolate all GPIO pins in sleep state␛[0m
␛[0;32mI (281) sleep: Enable automatic switching of GPIO sleep configuration␛[0m
␛[0;32mI (289) cpu_start: Starting scheduler on PRO CPU.␛[0m
␛[0;32mI (0) cpu_start: Starting scheduler on APP CPU.␛[0m
␛[0;32mI (309) gpio: GPIO[17]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0 ␛[0m

assert failed: 0x4037c261

Backtrace: 0x403756a2:0x3fc90f40 0x40379b45:0x3fc90f60 0x4037eada:0x3fc90f80 0x4037c261:0x3fc90fc0 0x4037a4c8:0x3fc90fe0 0x4200147d:0x3fc91010 0x420024b9:0x3fc91050 0x42002541:0x3fc91090 0x420063bf:0x3fc910d0 0x40376491:0x3fc91110 0x42012f93:0x3fcf3f70 0x420071e2:0x3fcf3f90 0x4037ab94:0x3fcf3fb0

ELF file SHA256: 30b5c0ed93c53172

Rebooting...
␀�ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
... loop forever```
lyusupov commented 1 year ago

It looks like an issue that affects the TinyUF2 & Actions infrastructure only. Manual builds ( self-hosted or hosted by Codespaces ) are still doing good to me.

SukuWc commented 1 year ago

I checked the assert address, it wasn't really helpful...

spinlock_acquire at /opt/esp/idf/components/esp_hw_support/include/soc/spinlock.h:75
(inlined by) xPortEnterCriticalTimeout at /opt/esp/idf/components/freertos/port/xtensa/port.c:293 

see also the full backtrace:

0x403756b2: panic_abort at /opt/esp/idf/components/esp_system/panic.c:408

0x40379b71: esp_system_abort at /opt/esp/idf/components/esp_system/esp_system.c:128

0x4037edfe: __assert_func at /opt/esp/idf/components/newlib/assert.c:47

0x4037c28d: spinlock_acquire at /opt/esp/idf/components/esp_hw_support/include/soc/spinlock.h:75
 (inlined by) xPortEnterCriticalTimeout at /opt/esp/idf/components/freertos/port/xtensa/port.c:293

0x4037a4f4: vPortEnterCritical at /opt/esp/idf/components/freertos/port/xtensa/include/freertos/portmacro.h:578
 (inlined by) xQueueGenericSendFromISR at /opt/esp/idf/components/freertos/queue.c:1082

0x420026d1: dcd_event_handler at ??:?

0x42006763: _dcd_int_handler at dcd_esp32sx.c:?

0x403764bd: _xt_lowint1 at /opt/esp/idf/components/freertos/port/xtensa/xtensa_vectors.S:1114

0x42012c63: cpu_ll_waiti at /opt/esp/idf/components/hal/esp32s3/include/hal/cpu_ll.h:182
 (inlined by) esp_pm_impl_waiti at /opt/esp/idf/components/esp_pm/pm_impl.c:837

0x4200763e: esp_vApplicationIdleHook at /opt/esp/idf/components/esp_system/freertos_hooks.c:63

0x4037abc0: prvIdleTask at /opt/esp/idf/components/freertos/tasks.c:3987

I am pretty sure nothing wrong with the source, the most likely culprit is still the idf docker image... Is there a way to get in touch with the espressif developers?

rgrizzell commented 1 year ago

The following is from my ESP32-S3-DevKitC1-N32R8V. I've been struggling to get just about any UF2 implementation onto this device.

I've tried:

These are the commands that gets me the closest to a running device. The one difference from the ports/espressif/README.md is that the bootloader.bin starts at 0x0 instead of 0x1000.

tinyuf2.bin esptool.py --chip esp32s3 -p /dev/ttyACM0 -b 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size detect 0x0 bootloader.bin 0x8000 partition-table.bin 0xe000 ota_data_initial.bin 0x2d0000 tinyuf2.bin

Show output ``` esptool.py v4.4 Serial port /dev/ttyACM0 Connecting... Chip is ESP32-S3 (revision v0.1) Features: WiFi, BLE Crystal is 40MHz MAC: 68:b6:b3:3d:7a:88 Uploading stub... Running stub... Stub running... Changing baud rate to 460800 Changed. Configuring flash size... WARNING: Flasher stub doesn't fully support flash size larger than 16MB, in case of failure use --no-stub. Flash will be erased from 0x00008000 to 0x00008fff... Flash will be erased from 0x0000e000 to 0x0000ffff... Flash will be erased from 0x00000000 to 0x00005fff... Flash will be erased from 0x00410000 to 0x00434fff... Compressed 3072 bytes to 136... Wrote 3072 bytes (136 compressed) at 0x00008000 in 0.0 seconds (effective 927.1 kbit/s)... Hash of data verified. Compressed 8192 bytes to 31... Wrote 8192 bytes (31 compressed) at 0x0000e000 in 0.1 seconds (effective 1213.7 kbit/s)... Hash of data verified. Warning: Image file at 0x0 is protected with a hash checksum, so not changing the flash size setting. Use the --flash_size=keep option instead of --flash_size=32MB in order to remove this warning, or use the --dont-append-digest option for the elf2image command in order to generate an image file without a hash checksum Compressed 22256 bytes to 14038... Wrote 22256 bytes (14038 compressed) at 0x00000000 in 0.2 seconds (effective 902.3 kbit/s)... Hash of data verified. Compressed 149248 bytes to 94705... Wrote 149248 bytes (94705 compressed) at 0x00410000 in 0.9 seconds (effective 1367.9 kbit/s)... Hash of data verified. Leaving... Hard resetting via RTS pin... ```

combined.bin esptool.py --chip esp32s3 -p /dev/ttyACM0 -b 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size detect 0x0 combined.bin

Show output ``` esptool.py v4.4 Serial port /dev/ttyACM0 Connecting... Chip is ESP32-S3 (revision v0.1) Features: WiFi, BLE Crystal is 40MHz MAC: 68:b6:b3:3d:7a:88 Uploading stub... Running stub... Stub running... Changing baud rate to 460800 Changed. Configuring flash size... WARNING: Flasher stub doesn't fully support flash size larger than 16MB, in case of failure use --no-stub. Flash will be erased from 0x00000000 to 0x00434fff... Warning: Image file at 0x0 is protected with a hash checksum, so not changing the flash size setting. Use the --flash_size=keep option instead of --flash_size=32MB in order to remove this warning, or use the --dont-append-digest option for the elf2image command in order to generate an image file without a hash checksum Compressed 4409088 bytes to 114796... Wrote 4409088 bytes (114796 compressed) at 0x00000000 in 20.9 seconds (effective 1688.6 kbit/s)... Hash of data verified. Leaving... Hard resetting via RTS pin... ```

UART Both flashing methods result in identical output.

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40375378
SPIWP:0xee
Octal Flash Mode Enabled
For OPI Flash, Use Default Flash Boot Mode
mode:SLOW_RD, clock div:1
load:0x3fcd0108,len:0x15ec
load:0x403b6000,len:0xdb8
load:0x403ba000,len:0x31c8
entry 0x403b6318
I (37) boot: ESP-IDF v4.4.2-378-g9269a536ac 2nd stage bootloader
I (38) boot: compile time 11:36:36
I (38) boot: chip revision: 0
I (41) qio_mode: Enabling default flash chip QIO
E (46) qio_mode: Failed to set QIE bit, not enabling QIO mode
I (52) boot.esp32s3: Boot SPI Speed : 80MHz
I (57) boot.esp32s3: SPI Mode       : SLOW READ
I (62) boot.esp32s3: SPI Flash Size : 8MB
I (67) boot: Enabling RNG early entropy source...
I (72) boot: Partition Table:
I (76) boot: ## Label            Usage          Type ST Offset   Length
I (83) boot:  0 nvs              WiFi data        01 02 00009000 00005000
I (91) boot:  1 otadata          OTA data         01 00 0000e000 00002000
I (98) boot:  2 ota_0            OTA app          00 10 00010000 00200000
I (106) boot:  3 ota_1            OTA app          00 11 00210000 00200000
I (113) boot:  4 uf2              factory app      00 00 00410000 00040000
I (121) boot:  5 ffat             Unknown data     01 81 00450000 003b0000
I (128) boot: End of partition table
I (133) boot: Defaulting to factory image
I (137) esp_image: segment 0: paddr=00410020 vaddr=3c020020 size=03dcch ( 15820) map
I (150) esp_image: segment 1: paddr=00413df4 vaddr=3fc90150 size=01c4ch (  7244) load
I (156) esp_image: segment 2: paddr=00415a48 vaddr=40374000 size=0a5d0h ( 42448) load
I (174) esp_image: segment 3: paddr=00420020 vaddr=42000020 size=1347ch ( 78972) map
I (193) esp_image: segment 4: paddr=004334a4 vaddr=4037e5d0 size=01b7ch (  7036) load
I (196) esp_image: segment 5: paddr=00435028 vaddr=50000000 size=00010h (    16) load
I (204) boot: Loaded app from partition at offset 0x410000
I (205) boot: Disabling RNG early entropy source...
I (222) cpu_start: Pro cpu up.
I (222) cpu_start: Starting app cpu, entry point is 0x40374d14
I (195) cpu_start: App cpu up.
I (237) cpu_start: Pro cpu start user code
I (237) cpu_start: cpu freq: 160000000
I (237) cpu_start: Application information:
I (239) cpu_start: Project name:     tinyuf2
I (244) cpu_start: App version:      0.11.0
I (249) cpu_start: Compile time:     Oct 17 2022 11:36:31
I (255) cpu_start: ELF file SHA256:  148a4f7802a8f23d...
I (261) cpu_start: ESP-IDF:          v4.4.2-378-g9269a536ac
I (268) heap_init: Initializing. RAM available for dynamic allocation:
I (275) heap_init: At 3FCA5478 len 00044298 (272 KiB): D/IRAM
I (281) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM
I (288) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM
I (294) heap_init: At 600FE000 len 00002000 (8 KiB): RTCRAM
I (301) spi_flash: detected chip: mxic
                                      W (305) spi_flash: Detected flash size > 16 MB, but access beyond 16 MB is not supported for this flash model yet.
          I (316) spi_flash: flash io: qio

assert failed: 0x42006b45

Backtrace: 0x4037564a:0x3fceb250 0x40379991:0x3fceb270 0x4037e926:0x3fceb290 0x42006b45:0x3fceb2d0 0x40374ff5:0x3fceb310 0x403bb410:0x3fceb340 0x403bb879:0x3fceb380 0x403b63c2:0x3fceb4b0 0x40045c01:0x3fceb570 |<-CORRUPTED

ELF file SHA256: 148a4f7802a8f23d

Rebooting...
rgrizzell commented 1 year ago

ESP32-S3-DevKitC1-N32R8V (WROOM-2)

Setup

My next approach was to compile from source. I started by defining a new board with the associated files. In this instance, I defined it as espressif_esp32s3_devkitc_1-1 and under it, I included the following files.

partitions-32MB.csv

# ESP-IDF Partition Table
# Name,   Type, SubType, Offset,  Size, Flags
# bootloader.bin,,          0x1000, 32K
# partition table,,         0x8000, 4K
nvs,      data, nvs,      0x9000,  20K,
otadata,  data, ota,      0xe000,  8K,
ota_0,    0,    ota_0,   0x10000,  2048K,
ota_1,    0,    ota_1,  0x210000,  2048K,
uf2,      app,  factory,0x410000,  256K,
ffat,     data, fat,    0x450000,  28352K,

sdkconfig

# Board Specific Config

# Partition Table
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions-32MB.csv"

# Serial flasher config
CONFIG_ESPTOOLPY_FLASHSIZE_32MB=y

board.h I have seen a few comments that on the WROOM-2, the Neopixel pin is GPIO38 instead of GPIO48.

# Apply board specific content here
set(IDF_TARGET "esp32s3")
/* 
 * The MIT License (MIT)
 *
 * Copyright (c) 2020 Ha Thach (tinyusb.org) for Adafruit Industries
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 */

#ifndef ESPRESSIF_S3_DEVKITC_H_
#define ESPRESSIF_S3_DEVKITC_H_

//--------------------------------------------------------------------+
// Button
//--------------------------------------------------------------------+

// Enter UF2 mode if GPIO is pressed while 2nd stage bootloader indicator
// is on e.g RGB = Purple. If it is GPIO0, user should not hold this while
// reset since that will instead run the 1st stage ROM bootloader
#define PIN_BUTTON_UF2        0

// GPIO that implement 1-bit memory with RC components which hold the
// pin value long enough for double reset detection.
// #define PIN_DOUBLE_RESET_RC

//--------------------------------------------------------------------+
// LED
//--------------------------------------------------------------------+

// GPIO connected to Neopixel data
#define NEOPIXEL_PIN          38

// Brightness percentage from 1 to 255
#define NEOPIXEL_BRIGHTNESS   0x10

// Number of neopixels
#define NEOPIXEL_NUMBER       1

// LED for indicator
// If not defined neopixel will be use for flash writing instead
// #define LED_PIN               42
// #define LED_STATE_ON          1

//--------------------------------------------------------------------+
// USB UF2
//--------------------------------------------------------------------+

#define USB_VID           0x239A
#define USB_PID           0x00A5    // TODO temporarily shared with S2 saola wrover
#define USB_MANUFACTURER  "Espressif"
#define USB_PRODUCT       "ESP32S3 DevKitC 1"

#define UF2_PRODUCT_NAME  USB_MANUFACTURER " " USB_PRODUCT
#define UF2_BOARD_ID      "ESP32S3-DevKitC-v1.1"
#define UF2_VOLUME_LABEL  "S3DKC1BOOT"
#define UF2_INDEX_URL     "https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/esp32s3/user-guide-devkitc-1.html"

#endif
# Board Specific Config

# Partition Table
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions-32MB.csv"

# Serial flasher config
CONFIG_ESPTOOLPY_FLASHSIZE_32MB=y

Compile

When I tried to compile I ran into issues generating the partitions tables. esptool.py (v3.2-dev) does not support flash sizes higher than 16.

make BOARD=espressif_esp32s3_devkitc_1-1 all

Show output ``` idf.py -B_build/espressif_esp32s3_devkitc_1-1 -DBOARD=espressif_esp32s3_devkitc_1-1 build Executing action: all (aliases: build) Running ninja in directory /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1 Executing "ninja all"... [0/1] Re-running CMake...-- Building ESP-IDF components for target esp32s3 -- Project sdkconfig file /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/sdkconfig Loading defaults file /home/rgrizzell/tinyuf2/ports/espressif/sdkconfig.defaults... /home/rgrizzell/tinyuf2/ports/espressif/sdkconfig.defaults:21 CONFIG_FLASHMODE_QIO was replaced with CONFIG_ESPTOOLPY_FLASHMODE_QIO Loading defaults file /home/rgrizzell/tinyuf2/ports/espressif/boards/espressif_esp32s3_devkitc_1-1/sdkconfig... -- Found Python3: /home/rwgrizzell/.espressif/python_env/idf5.0_py3.10_env/bin/python (found version "3.10.6") found components: Interpreter -- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS -- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS - Success -- App "tinyuf2" version: 0.11.0-dirty -- Adding linker script /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/esp-idf/esp_system/ld/memory.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_system/ld/esp32s3/sections.ld.in -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.api.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.libgcc.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.newlib.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.version.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/soc/esp32s3/ld/esp32s3.peripherals.ld -- Components: app_trace app_update boards bootloader bootloader_support bt cmock console cxx driver efuse esp-tls esp_adc esp_app_format esp_common esp_eth esp_event esp_gdbstub esp_hid esp_http_client esp_http_server esp_https_ota esp_https_server esp_hw_support esp_lcd esp_local_ctrl esp_netif esp_partition esp_phy esp_pm esp_psram esp_ringbuf esp_rom esp_system esp_timer esp_wifi espcoredump esptool_py fatfs freertos hal heap http_parser idf_test ieee802154 json lcd led_strip log lwip mbedtls mqtt newlib nvs_flash openthread partition_table perfmon protobuf-c protocomm pthread sdmmc soc spi_flash spiffs src tcp_transport touch_element ulp unity usb vfs wear_levelling wifi_provisioning wpa_supplicant xtensa -- Component paths: /home/rgrizzell/tinyuf2/lib/esp-idf/components/app_trace /home/rgrizzell/tinyuf2/lib/esp-idf/components/app_update /home/rgrizzell/tinyuf2/ports/espressif/boards /home/rgrizzell/tinyuf2/ports/espressif/components/bootloader /home/rgrizzell/tinyuf2/lib/esp-idf/components/bootloader_support /home/rgrizzell/tinyuf2/lib/esp-idf/components/bt /home/rgrizzell/tinyuf2/lib/esp-idf/components/cmock /home/rgrizzell/tinyuf2/lib/esp-idf/components/console /home/rgrizzell/tinyuf2/lib/esp-idf/components/cxx /home/rgrizzell/tinyuf2/lib/esp-idf/components/driver /home/rgrizzell/tinyuf2/lib/esp-idf/components/efuse /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp-tls /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_adc /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_app_format /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_common /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_eth /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_event /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_gdbstub /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_hid /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_http_client /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_http_server /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_https_ota /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_https_server /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_hw_support /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_lcd /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_local_ctrl /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_netif /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_partition /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_phy /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_pm /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_psram /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_ringbuf /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_system /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_timer /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_wifi /home/rgrizzell/tinyuf2/lib/esp-idf/components/espcoredump /home/rgrizzell/tinyuf2/lib/esp-idf/components/esptool_py /home/rgrizzell/tinyuf2/lib/esp-idf/components/fatfs /home/rgrizzell/tinyuf2/lib/esp-idf/components/freertos /home/rgrizzell/tinyuf2/lib/esp-idf/components/hal /home/rgrizzell/tinyuf2/lib/esp-idf/components/heap /home/rgrizzell/tinyuf2/lib/esp-idf/components/http_parser /home/rgrizzell/tinyuf2/lib/esp-idf/components/idf_test /home/rgrizzell/tinyuf2/lib/esp-idf/components/ieee802154 /home/rgrizzell/tinyuf2/lib/esp-idf/components/json /home/rgrizzell/tinyuf2/ports/espressif/components/lcd /home/rgrizzell/tinyuf2/ports/espressif/components/led_strip /home/rgrizzell/tinyuf2/lib/esp-idf/components/log /home/rgrizzell/tinyuf2/lib/esp-idf/components/lwip /home/rgrizzell/tinyuf2/lib/esp-idf/components/mbedtls /home/rgrizzell/tinyuf2/lib/esp-idf/components/mqtt /home/rgrizzell/tinyuf2/lib/esp-idf/components/newlib /home/rgrizzell/tinyuf2/lib/esp-idf/components/nvs_flash /home/rgrizzell/tinyuf2/lib/esp-idf/components/openthread /home/rgrizzell/tinyuf2/lib/esp-idf/components/partition_table /home/rgrizzell/tinyuf2/lib/esp-idf/components/perfmon /home/rgrizzell/tinyuf2/lib/esp-idf/components/protobuf-c /home/rgrizzell/tinyuf2/lib/esp-idf/components/protocomm /home/rgrizzell/tinyuf2/lib/esp-idf/components/pthread /home/rgrizzell/tinyuf2/lib/esp-idf/components/sdmmc /home/rgrizzell/tinyuf2/lib/esp-idf/components/soc /home/rgrizzell/tinyuf2/lib/esp-idf/components/spi_flash /home/rgrizzell/tinyuf2/lib/esp-idf/components/spiffs /home/rgrizzell/tinyuf2/src /home/rgrizzell/tinyuf2/lib/esp-idf/components/tcp_transport /home/rgrizzell/tinyuf2/lib/esp-idf/components/touch_element /home/rgrizzell/tinyuf2/lib/esp-idf/components/ulp /home/rgrizzell/tinyuf2/lib/esp-idf/components/unity /home/rgrizzell/tinyuf2/lib/esp-idf/components/usb /home/rgrizzell/tinyuf2/lib/esp-idf/components/vfs /home/rgrizzell/tinyuf2/lib/esp-idf/components/wear_levelling /home/rgrizzell/tinyuf2/lib/esp-idf/components/wifi_provisioning /home/rgrizzell/tinyuf2/lib/esp-idf/components/wpa_supplicant /home/rgrizzell/tinyuf2/lib/esp-idf/components/xtensa -- Configuring done -- Generating done -- Build files have been written to: /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1 [4/917] Generating ../../partition_table/partition-table.binFAILED: partition_table/partition-table.bin /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/partition_table/partition-table.bin cd /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/esp-idf/partition_table && /home/rwgrizzell/.espressif/python_env/idf5.0_py3.10_env/bin/python /home/rgrizzell/tinyuf2/lib/esp-idf/components/partition_table/gen_esp32part.py -q --offset 0x8000 --flash-size 2MB -- /home/rgrizzell/tinyuf2/ports/espressif/partitions-32MB.csv /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/partition_table/partition-table.bin && /usr/bin/cmake -E echo "Partition table binary generated. Contents:" && /usr/bin/cmake -E echo "*******************************************************************************" && /home/rwgrizzell/.espressif/python_env/idf5.0_py3.10_env/bin/python /home/rgrizzell/tinyuf2/lib/esp-idf/components/partition_table/gen_esp32part.py -q --offset 0x8000 --flash-size 2MB -- /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/partition_table/partition-table.bin && /usr/bin/cmake -E echo "*******************************************************************************" Partitions tables occupies 32.0MB of flash (33554432 bytes) which does not fit in configured flash size 2MB. Change the flash size in menuconfig under the 'Serial Flasher Config' menu. [37/917] Building C object esp-idf/mbedtls/mbedtls/library/CMakeFiles/mbedcrypto.dir/bignum.c.objninja: build stopped: subcommand failed. ninja failed with exit code 1, output of the command is in the /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/log/idf_py_stderr_output_748177 and /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/log/idf_py_stdout_output_748177 make: *** [Makefile:31: all] Error 2 ```

I then updated the esp-idf submodule to v5.0 (which updates esptool.py to v4.4), the Serial Flasher then supports 32MB, recognizing the variables set in sdkconfig. Confirmed by running: idf.py menuconfig -DBOARD=espressif_esp32s3_devkitc_1-1

Show output ``` idf.py -B_build/espressif_esp32s3_devkitc_1-1 -DBOARD=espressif_esp32s3_devkitc_1-1 build Executing action: all (aliases: build) Running cmake in directory /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1 Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 -DBOARD=espressif_esp32s3_devkitc_1-1 -DCCACHE_ENABLE=0 /home/rgrizzell/tinyuf2/ports/espressif"... -- Found Git: /usr/bin/git (found version "2.34.1") -- The C compiler identification is GNU 11.2.0 -- The CXX compiler identification is GNU 11.2.0 -- The ASM compiler identification is GNU -- Found assembler: /home/rgrizzell/.espressif/tools/xtensa-esp32s3-elf/esp-2022r1-11.2.0/xtensa-esp32s3-elf/bin/xtensa-esp32s3-elf-gcc -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /home/rgrizzell/.espressif/tools/xtensa-esp32s3-elf/esp-2022r1-11.2.0/xtensa-esp32s3-elf/bin/xtensa-esp32s3-elf-gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /home/rgrizzell/.espressif/tools/xtensa-esp32s3-elf/esp-2022r1-11.2.0/xtensa-esp32s3-elf/bin/xtensa-esp32s3-elf-g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Building ESP-IDF components for target esp32s3 -- Project sdkconfig file /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/sdkconfig Loading defaults file /home/rgrizzell/tinyuf2/ports/espressif/sdkconfig.defaults... /home/rgrizzell/tinyuf2/ports/espressif/sdkconfig.defaults:21 CONFIG_FLASHMODE_QIO was replaced with CONFIG_ESPTOOLPY_FLASHMODE_QIO Loading defaults file /home/rgrizzell/tinyuf2/ports/espressif/boards/espressif_esp32s3_devkitc_1-1/sdkconfig... -- Looking for sys/types.h -- Looking for sys/types.h - found -- Looking for stdint.h -- Looking for stdint.h - found -- Looking for stddef.h -- Looking for stddef.h - found -- Check size of time_t -- Check size of time_t - done -- Found Python3: /home/rgrizzell/.espressif/python_env/idf5.0_py3.10_env/bin/python (found version "3.10.6") found components: Interpreter -- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS -- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS - Success -- App "tinyuf2" version: 0.11.0-dirty -- Adding linker script /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/esp-idf/esp_system/ld/memory.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_system/ld/esp32s3/sections.ld.in -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.api.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.libgcc.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.newlib.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom/esp32s3/ld/esp32s3.rom.version.ld -- Adding linker script /home/rgrizzell/tinyuf2/lib/esp-idf/components/soc/esp32s3/ld/esp32s3.peripherals.ld -- Components: app_trace app_update boards bootloader bootloader_support bt cmock console cxx driver efuse esp-tls esp_adc esp_app_format esp_common esp_eth esp_event esp_gdbstub esp_hid esp_http_client esp_http_server esp_https_ota esp_https_server esp_hw_support esp_lcd esp_local_ctrl esp_netif esp_partition esp_phy esp_pm esp_psram esp_ringbuf esp_rom esp_system esp_timer esp_wifi espcoredump esptool_py fatfs freertos hal heap http_parser idf_test ieee802154 json lcd led_strip log lwip mbedtls mqtt newlib nvs_flash openthread partition_table perfmon protobuf-c protocomm pthread sdmmc soc spi_flash spiffs src tcp_transport touch_element ulp unity usb vfs wear_levelling wifi_provisioning wpa_supplicant xtensa -- Component paths: /home/rgrizzell/tinyuf2/lib/esp-idf/components/app_trace /home/rgrizzell/tinyuf2/lib/esp-idf/components/app_update /home/rgrizzell/tinyuf2/ports/espressif/boards /home/rgrizzell/tinyuf2/ports/espressif/components/bootloader /home/rgrizzell/tinyuf2/lib/esp-idf/components/bootloader_support /home/rgrizzell/tinyuf2/lib/esp-idf/components/bt /home/rgrizzell/tinyuf2/lib/esp-idf/components/cmock /home/rgrizzell/tinyuf2/lib/esp-idf/components/console /home/rgrizzell/tinyuf2/lib/esp-idf/components/cxx /home/rgrizzell/tinyuf2/lib/esp-idf/components/driver /home/rgrizzell/tinyuf2/lib/esp-idf/components/efuse /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp-tls /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_adc /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_app_format /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_common /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_eth /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_event /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_gdbstub /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_hid /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_http_client /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_http_server /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_https_ota /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_https_server /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_hw_support /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_lcd /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_local_ctrl /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_netif /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_partition /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_phy /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_pm /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_psram /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_ringbuf /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_rom /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_system /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_timer /home/rgrizzell/tinyuf2/lib/esp-idf/components/esp_wifi /home/rgrizzell/tinyuf2/lib/esp-idf/components/espcoredump /home/rgrizzell/tinyuf2/lib/esp-idf/components/esptool_py /home/rgrizzell/tinyuf2/lib/esp-idf/components/fatfs /home/rgrizzell/tinyuf2/lib/esp-idf/components/freertos /home/rgrizzell/tinyuf2/lib/esp-idf/components/hal /home/rgrizzell/tinyuf2/lib/esp-idf/components/heap /home/rgrizzell/tinyuf2/lib/esp-idf/components/http_parser /home/rgrizzell/tinyuf2/lib/esp-idf/components/idf_test /home/rgrizzell/tinyuf2/lib/esp-idf/components/ieee802154 /home/rgrizzell/tinyuf2/lib/esp-idf/components/json /home/rgrizzell/tinyuf2/ports/espressif/components/lcd /home/rgrizzell/tinyuf2/ports/espressif/components/led_strip /home/rgrizzell/tinyuf2/lib/esp-idf/components/log /home/rgrizzell/tinyuf2/lib/esp-idf/components/lwip /home/rgrizzell/tinyuf2/lib/esp-idf/components/mbedtls /home/rgrizzell/tinyuf2/lib/esp-idf/components/mqtt /home/rgrizzell/tinyuf2/lib/esp-idf/components/newlib /home/rgrizzell/tinyuf2/lib/esp-idf/components/nvs_flash /home/rgrizzell/tinyuf2/lib/esp-idf/components/openthread /home/rgrizzell/tinyuf2/lib/esp-idf/components/partition_table /home/rgrizzell/tinyuf2/lib/esp-idf/components/perfmon /home/rgrizzell/tinyuf2/lib/esp-idf/components/protobuf-c /home/rgrizzell/tinyuf2/lib/esp-idf/components/protocomm /home/rgrizzell/tinyuf2/lib/esp-idf/components/pthread /home/rgrizzell/tinyuf2/lib/esp-idf/components/sdmmc /home/rgrizzell/tinyuf2/lib/esp-idf/components/soc /home/rgrizzell/tinyuf2/lib/esp-idf/components/spi_flash /home/rgrizzell/tinyuf2/lib/esp-idf/components/spiffs /home/rgrizzell/tinyuf2/src /home/rgrizzell/tinyuf2/lib/esp-idf/components/tcp_transport /home/rgrizzell/tinyuf2/lib/esp-idf/components/touch_element /home/rgrizzell/tinyuf2/lib/esp-idf/components/ulp /home/rgrizzell/tinyuf2/lib/esp-idf/components/unity /home/rgrizzell/tinyuf2/lib/esp-idf/components/usb /home/rgrizzell/tinyuf2/lib/esp-idf/components/vfs /home/rgrizzell/tinyuf2/lib/esp-idf/components/wear_levelling /home/rgrizzell/tinyuf2/lib/esp-idf/components/wifi_provisioning /home/rgrizzell/tinyuf2/lib/esp-idf/components/wpa_supplicant /home/rgrizzell/tinyuf2/lib/esp-idf/components/xtensa -- Configuring done -- Generating done -- Build files have been written to: /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1 Running ninja in directory /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1 Executing "ninja all"... [28/917] Generating ../../partition_table/partition-table.binPartition table binary generated. Contents: ******************************************************************************* # ESP-IDF Partition Table # Name, Type, SubType, Offset, Size, Flags nvs,data,nvs,0x9000,20K, otadata,data,ota,0xe000,8K, ota_0,app,ota_0,0x10000,2M, ota_1,app,ota_1,0x210000,2M, uf2,app,factory,0x410000,256K, ffat,data,fat,0x450000,28352K, ******************************************************************************* [398/917] Performing configure step for 'bootloader'FAILED: bootloader-prefix/src/bootloader-stamp/bootloader-configure /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/bootloader-prefix/src/bootloader-stamp/bootloader-configure cd /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/bootloader && /usr/bin/cmake -DSDKCONFIG=/home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/sdkconfig -DIDF_PATH=/home/rgrizzell/tinyuf2/lib/esp-idf -DIDF_TARGET=esp32s3 -DPYTHON_DEPS_CHECKED=1 -DPYTHON=/home/rgrizzell/.espressif/python_env/idf5.0_py3.10_env/bin/python -DEXTRA_COMPONENT_DIRS=/home/rgrizzell/tinyuf2/ports/espressif/components/bootloader -DPROJECT_SOURCE_DIR=/home/rgrizzell/tinyuf2/ports/espressif -DBOARD=espressif_esp32s3_devkitc_1-1 -DLEGACY_INCLUDE_COMMON_HEADERS= -GNinja /home/rgrizzell/tinyuf2/ports/espressif/components/bootloader/subproject && /usr/bin/cmake -E touch /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/bootloader-prefix/src/bootloader-stamp/bootloader-configure -- Found Git: /usr/bin/git (found version "2.34.1") CMake Error at /home/rgrizzell/tinyuf2/lib/esp-idf/tools/cmake/project.cmake:243 (message): Directory specified in EXTRA_COMPONENT_DIRS doesn't exist: /home/rgrizzell/tinyuf2/ports/espressif/bootloader_components Call Stack (most recent call first): /home/rgrizzell/tinyuf2/lib/esp-idf/tools/cmake/project.cmake:327 (__project_init) CMakeLists.txt:59 (project) -- Configuring incomplete, errors occurred! [431/917] Building CXX object esp-idf/nvs_flash/CMakeFiles/__idf_nvs_flash.dir/src/nvs_storage.cpp.objninja: build stopped: subcommand failed. ninja failed with exit code 1, output of the command is in the /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/log/idf_py_stderr_output_799291 and /home/rgrizzell/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1-1/log/idf_py_stdout_output_799291 make: *** [Makefile:31: all] Error 2 ```

At this point, it seems there are some breaking changes between ESP-IDF v4.4 and v5.0.

SukuWc commented 1 year ago

Migrating to V5.0 would be the best but also the most effort in my opinion. In the meanwhile I contacted espressif support directly, hopefully they can help us out!

hathach commented 1 year ago

I don't have any unexpected maker boards to test this with. Can you confirm this issue on one of Adafruit or Espressif following boards:

skieast commented 1 year ago

I built tinyuf2 for the espressif_esp32s3_box from master today and it does work.
Also built for espressif_esp32s3_devkitc_1 which didn't work. No idea why as the hardware should be essentially the same. The devkit has 32MB of flash but i tried building for 8MB and 16MB, Can flash both but still gets a bootloop. Tried a few different idf versions incliuding the latest 4.4.3.

skieast commented 1 year ago

I changed the sdkconfig.defaults file as follows

# Compatibility options
#CONFIG_FLASHMODE_QIO=y
CONFIG_FLASHMODE_DIO=y

And built for the espressif_esp32s3_devkitc_1 and the unexpectedmaker_feathers3 and both worked. I don't have time tonight to further test and i have made a lot of test changes. Will relclone the repository tomorrow and see if this all still works.

lyusupov commented 1 year ago

@skieast it was reported earlier that there is nothing wrong with self-hosted builds.

The original issue report is about regression of Actions build since November 15th. https://github.com/adafruit/tinyuf2/actions/runs/3469908233 The build deploys certain 'artifacts' that don't boot on an actual ESP32-Sx hardware.

image

lyusupov commented 1 year ago

I don't have any unexpected maker boards to test this with. Can you confirm this issue on one of Adafruit or Espressif following boards:

  • 'adafruit_feather_esp32s3'
  • 'adafruit_feather_esp32s3_nopsram'
  • 'adafruit_feather_esp32s3_tft'
  • 'adafruit_qtpy_esp32s3'
  • 'espressif_esp32s3_devkitc_1'
  • 'espressif_esp32s3_devkitm_1'

@hathach in order to reproduce the issue, please, try to boot a firmware binary from 'Artifacts' section of this page: https://github.com/adafruit/tinyuf2/actions/runs/3469908233 on any of the boards from your list.

skieast commented 1 year ago

@skieast it was reported earlier that there is nothing wrong with self-hosted builds.

The original issue report is about regression of Actions build since November 15th. https://github.com/adafruit/tinyuf2/actions/runs/3469908233 The build deploys certain 'artifacts' that don't boot on an actual ESP32-Sx hardware.

image

Don't think this is the same issue as the one you are referencing. I only use self-hosted builds and can confirm that building for some esp32s3 boards does not work and results in a bootloop. Changing the flashmode in sdkconfig to DIO for the builds in question appears to fix this. I had planned to look at this again today to confirm. I have several of the boards in question so can test.

skieast commented 1 year ago

Just recloned the repository, rebooted my laptop and did builds for the espressif_esp32s3_box, espressif_esp32s3_devkitc_1 (32mb flash), unexpectedmaker_tinys3, and unexpectedmaker_feathers3 . All flashed and booted with an enumerated drive. Using Linux to build and test. I had an issue with the espressif_esp32s3_devkitc_1 but I bvelieve that was due to the 32mb flash and that its octal. Changing the sdkconfig file to include the line CONFIG_FLASHMODE_DIO=y for that board fixed that. So can't see what the original issues was. Or why it didnt work for me when i first tried it.

SukuWc commented 1 year ago

CONFIG_FLASHMODE_DIO=y did not help for me, it still bootloops on the TinyS3 compatible targets... (Build using GitHub Actions, idf:release_v4.4)

rgrizzell commented 1 year ago

Just recloned the repository, rebooted my laptop and did builds for the espressif_esp32s3_box, espressif_esp32s3_devkitc_1 (32mb flash), unexpectedmaker_tinys3, and unexpectedmaker_feathers3 . All flashed and booted with an enumerated drive. Using Linux to build and test. I had an issue with the espressif_esp32s3_devkitc_1 but I bvelieve that was due to the 32mb flash and that its octal. Changing the sdkconfig file to include the line CONFIG_FLASHMODE_DIO=y for that board fixed that. So can't see what the original issues was. Or why it didnt work for me when i first tried it.

This worked for me! 👍🏼

make BOARD=espressif_esp32s3_devkitc_1 all make BOARD=espressif_esp32s3_devkitc_1 flash

Output ``` idf.py -B_build/espressif_esp32s3_devkitc_1 -DBOARD=espressif_esp32s3_devkitc_1 flash Executing action: flash Serial port /dev/ttyUSB0 /dev/ttyUSB0 failed to connect: [Errno 16] could not open port /dev/ttyUSB0: [Errno 16] Device or resource busy: '/dev/ttyUSB0' Serial port /dev/ttyACM0 Connecting... Detecting chip type... ESP32-S3 Running ninja in directory /home/rgrizzell/Projects/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1 Executing "ninja flash"... [1/5] cd /home/rgrizzell/Projects/tinyuf2/ports/espressif/_build/esp...inyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1/tinyuf2.bin tinyuf2.bin binary size 0x24420 bytes. Smallest app partition is 0x40000 bytes. 0x1bbe0 bytes (43%) free. [2/5] Performing build step for 'bootloader' [1/1] cd /home/rgrizzell/Projects/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1/bootloader/esp-idf/esptool_py && /home/rgrizzell/.espressif/python_env/idf4.4_py3.10_env/bin/python /home/rgrizzell/Projects/tinyuf2/lib/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 bootloader 0x0 /home/rgrizzell/Projects/tinyuf2/ports/espressif/_build/espressif_esp32s3_devkitc_1/bootloader/bootloader.bin Bootloader binary size 0x4ec0 bytes. 0x3140 bytes (38%) free. [2/3] cd /home/rgrizzell/Projects/tinyuf2/lib/esp-idf/components/esp...jects/tinyuf2/lib/esp-idf/components/esptool_py/run_serial_tool.cmake esptool.py esp32s3 -p /dev/ttyACM0 -b 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size 8MB 0x0 bootloader/bootloader.bin 0x410000 tinyuf2.bin 0x8000 partition_table/partition-table.bin 0xe000 ota_data_initial.bin esptool.py v3.2-dev Serial port /dev/ttyACM0 Connecting... Chip is ESP32-S3 Features: WiFi, BLE Crystal is 40MHz MAC: 68:b6:b3:3d:7a:88 Uploading stub... Running stub... Stub running... Changing baud rate to 460800 Changed. Configuring flash size... Flash will be erased from 0x00000000 to 0x00004fff... Flash will be erased from 0x00410000 to 0x00434fff... Flash will be erased from 0x00008000 to 0x00008fff... Flash will be erased from 0x0000e000 to 0x0000ffff... Compressed 20160 bytes to 12799... Writing at 0x00000000... (100 %) Wrote 20160 bytes (12799 compressed) at 0x00000000 in 0.2 seconds (effective 726.4 kbit/s)... Hash of data verified. Compressed 148512 bytes to 94405... Writing at 0x00410000... (16 %) Writing at 0x00418a3c... (33 %) Writing at 0x0041e12f... (50 %) Writing at 0x00424fac... (66 %) Writing at 0x0042ac5b... (83 %) Writing at 0x0043012e... (100 %) Wrote 148512 bytes (94405 compressed) at 0x00410000 in 1.2 seconds (effective 1032.2 kbit/s)... Hash of data verified. Compressed 3072 bytes to 136... Writing at 0x00008000... (100 %) Wrote 3072 bytes (136 compressed) at 0x00008000 in 0.0 seconds (effective 770.3 kbit/s)... Hash of data verified. Compressed 8192 bytes to 31... Writing at 0x0000e000... (100 %) Wrote 8192 bytes (31 compressed) at 0x0000e000 in 0.1 seconds (effective 1019.1 kbit/s)... Hash of data verified. Leaving... Hard resetting via RTS pin... Done ```

I've got full use of the 32MB of flash. 💯

$ df -h /media/rgrizzell/S3DKC1BOOT/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda         32M  4.1M   28M  13% /media/rgrizzell/S3DKC1BOOT
rgrizzell commented 1 year ago

At this point I feel comfortable enough to say that issues with the ESP32-S3-DevKitC-1-N16R8V and ESP32-S3-DevKitC-1-N32R8V are separate from the reportedly broken builds in this thread.

skieast commented 1 year ago

At this point I feel comfortable enough to say that issues with the ESP32-S3-DevKitC-1-N16R8V and ESP32-S3-DevKitC-1-N32R8V are separate from the reportedly broken builds in this thread.

Agreed.

hathach commented 1 year ago

I don't have any unexpected maker boards to test this with. Can you confirm this issue on one of Adafruit or Espressif following boards:

  • 'adafruit_feather_esp32s3'
  • 'adafruit_feather_esp32s3_nopsram'
  • 'adafruit_feather_esp32s3_tft'
  • 'adafruit_qtpy_esp32s3'
  • 'espressif_esp32s3_devkitc_1'
  • 'espressif_esp32s3_devkitm_1'

@hathach in order to reproduce the issue, please, try to boot a firmware binary from 'Artifacts' section of this page: https://github.com/adafruit/tinyuf2/actions/runs/3469908233 on any of the boards from your list.

so far, this issue is only reproduced on tinyS3 board. I haven't seen any one confirmed it with one of above list yet.

skieast commented 1 year ago

I think the issue may be with the docker image of the esp-idf being used

run: docker run --rm -v $PWD:/project -w /project espressif/idf:release-v4.4  /bin/bash -c "git config --global --add safe.directory /project && make -C ports/espressif/ BOARD=${{ matrix.board }} all self-update copy-artifact"

When i try to build locally using the current release-v4.4 for the TinyS3 it does not work.
If I build locally using the version of esp-idf included in the tinyuf2 repository it does work.

skieast commented 1 year ago

So I built a new docker image for the esp-idf that uses the correct commit. Modified my fork to use this docker image. The artifacts for the tinys3 and feathers3 both enumerate now. I did not test any other boards.

Docker hub repo, image is v4.4-tinyuf2 : https://hub.docker.com/repository/docker/6lhasacoi/idf-custom Fork and branch to use this image: https://github.com/skieast/tinyuf2/tree/test-workflow Artifacts from last push: https://github.com/skieast/tinyuf2/actions/runs/3783255645

I have almost no knowledge of docker but managed to get something that worked.

SukuWc commented 1 year ago

@skieast thank you for your effort creating the custom docker image, I just tested it and it works for my custom (TinyS3 compatible) esp32s3 boards! What next? Should we try to convince espressif to host this image on their official docker hub? Maybe adafruit?

skieast commented 1 year ago

@SukuWc thanks for trying this and confirming that it works, at least for you and me, Now it's up to @hathach to decide on how to proceed. If this is the route to follow (custom docker image) then on some official docker hub repository I think. Or someone can find out why the 4.4 release bootloops with the esp32s3.

skieast commented 1 year ago

I also looked briefly at hosting the image at Github Packages repository which supports Docker images (amongst others). I didn't get all the permissions right to push my local image but it should work. This would then allow the idf-custom Docker image to be hosted in the github tinyuf2 reposity. Or maybe its the adafruit account. Smarter brains will know.

https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry

hathach commented 1 year ago

Custom docker image will not be accepted. If docker image is the issue, you should report it to espressif for them to fix. If it is the tinyuf2 we should fix it here. Though I don't have the boards with this issue, therefore pr is welcome

skieast commented 1 year ago

It's an image created using espressif tools designed to get an image at a defined commit. In this case the esp-idf in the tinyuf2 repository is at commit bbe2a1bf34. The docker idf image used in the github workflow is v4.4. This image changes as espressif updates that version. So you will get times where it may work depending on what espressif has committed to that image.
In any case thanks for looking at the research.

skieast commented 1 year ago

I have tested modifying the .github/workflows/build_esp32.yml to use the espressif/idf:v4.4.3 idf image. In this case the artifacts do enumerate on the boards I tested.

unexpectedmaker_feathers3
unexpectedmaker_tinys3
espressif_esp32s3_box
20230106 - Just tested with adafruit_feather_esp32s3 .

The artifacts I pulled from the master branch built a week ago did not work on the above boards. Perhaps this is an acceptable solution.

 git diff master                                                                          Mon Jan  2 17:09:50 2023
diff --git a/.github/workflows/build_esp32.yml b/.github/workflows/build_esp32.yml
index e14093db..18cdf7a7 100644
--- a/.github/workflows/build_esp32.yml
+++ b/.github/workflows/build_esp32.yml
@@ -99,7 +99,7 @@ jobs:
       run: echo BIN_PATH=ports/espressif/_bin/${{ matrix.board }} >> $GITHUB_ENV

     - name: Build
-      run: docker run --rm -v $PWD:/project -w /project espressif/idf:release-v4.4  /bin/bash -c "git config --global --add safe.directory /project && make -C ports/espressif/ BOARD=${{ matrix.board }} all self-update copy-artifact"
+      run: docker run --rm -v $PWD:/project -w /project espressif/idf:v4.4.3  /bin/bash -c "git config --global --add safe.directory /project && make -C ports/espressif/ BOARD=${{ matrix.board }} all self-update copy-artifact"

     - uses: actions/upload-artifact@v3
       with:

If I build locally using the release/v4.4 branch the result does not work on the unexpectedmaker_feathers3. If i build locally using the v4.4.3 tag of the idf it does work.

Artifacts from build: https://github.com/skieast/tinyuf2/actions/runs/3826076939

hathach commented 1 year ago

thank you @skieast for testing this out, temporarily switch to v4.4.3 (released 2 months ago), seems like the latest v4.4 (released 2 weeks ago) indeed is the troublesome one.

hathach commented 1 year ago

I have release 0.12.0 please try it out to see if that works for you.