Invalid header when using Secure Boot and Flash Encryption (IDFGH-9136)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v4.4.3-347-g9ee3c8337d

Operating System used.

macOS

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

What is the expected behavior?

Device to boot normally and perform flash encryption.

What is the actual behavior?

After burning bootloader and firmware get invalid header.

Steps to reproduce.

Following instructions from here: https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32s3/security/secure-boot-v2.html

Using secure boot signing key generated by espsecure.py generate_signing_key

Firmware works normally when not encrypted and when secure boot is not enabled.
Burning the signed bootloader and code produces invalid error header. I also burn certs to esp_secure_cert. I have tried burning these secure certs as the first operation and on a separate attempt as the last operation and still get the invalid header error.
```
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x7 (TG0WDT_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40048c3e
invalid header: 0x507bc083
invalid header: 0x507bc083
invalid header: 0x507bc083
```
I have tried implementing the changes suggested in #5771 and get the same results. See SDKconfig settings below.

SDKCONFIG

#
# Partition Table
#
# CONFIG_PARTITION_TABLE_SINGLE_APP is not set
# CONFIG_PARTITION_TABLE_SINGLE_APP_LARGE is not set
# CONFIG_PARTITION_TABLE_TWO_OTA is not set
CONFIG_PARTITION_TABLE_CUSTOM=y
# CONFIG_PARTITION_TABLE_SINGLE_APP_ENCRYPTED_NVS is not set
# CONFIG_PARTITION_TABLE_SINGLE_APP_LARGE_ENC_NVS is not set
# CONFIG_PARTITION_TABLE_TWO_OTA_ENCRYPTED_NVS is not set
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_ota.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions_ota.csv"
CONFIG_PARTITION_TABLE_OFFSET=0xA000
CONFIG_PARTITION_TABLE_MD5=y
# end of Partition Table

#
# Bootloader config
#
CONFIG_BOOTLOADER_OFFSET_IN_FLASH=0x0
CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_SIZE=y
# CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_DEBUG is not set
# CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_PERF is not set
# CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_NONE is not set
# CONFIG_BOOTLOADER_LOG_LEVEL_NONE is not set
CONFIG_BOOTLOADER_LOG_LEVEL_ERROR=y
# CONFIG_BOOTLOADER_LOG_LEVEL_WARN is not set
# CONFIG_BOOTLOADER_LOG_LEVEL_INFO is not set
# CONFIG_BOOTLOADER_LOG_LEVEL_DEBUG is not set
# CONFIG_BOOTLOADER_LOG_LEVEL_VERBOSE is not set
CONFIG_BOOTLOADER_LOG_LEVEL=1
CONFIG_BOOTLOADER_VDDSDIO_BOOST_1_9V=y
CONFIG_BOOTLOADER_FACTORY_RESET=y
CONFIG_BOOTLOADER_NUM_PIN_FACTORY_RESET=3
CONFIG_BOOTLOADER_FACTORY_RESET_PIN_LOW=y
# CONFIG_BOOTLOADER_FACTORY_RESET_PIN_HIGH is not set
CONFIG_BOOTLOADER_OTA_DATA_ERASE=y
CONFIG_BOOTLOADER_DATA_FACTORY_RESET="nvs"
# CONFIG_BOOTLOADER_APP_TEST is not set
CONFIG_BOOTLOADER_HOLD_TIME_GPIO=5
CONFIG_BOOTLOADER_REGION_PROTECTION_ENABLE=y
CONFIG_BOOTLOADER_WDT_ENABLE=y
# CONFIG_BOOTLOADER_WDT_DISABLE_IN_USER_CODE is not set
CONFIG_BOOTLOADER_WDT_TIME_MS=9000
CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y
# CONFIG_BOOTLOADER_APP_ANTI_ROLLBACK is not set
# CONFIG_BOOTLOADER_SKIP_VALIDATE_IN_DEEP_SLEEP is not set
CONFIG_BOOTLOADER_RESERVE_RTC_SIZE=0
# CONFIG_BOOTLOADER_CUSTOM_RESERVE_RTC is not set
CONFIG_BOOTLOADER_FLASH_XMC_SUPPORT=y
# end of Bootloader config

#
# Security features
#
CONFIG_SECURE_SIGNED_ON_BOOT=y
CONFIG_SECURE_SIGNED_ON_UPDATE=y
CONFIG_SECURE_SIGNED_APPS=y
CONFIG_SECURE_BOOT_SUPPORTS_RSA=y
CONFIG_SECURE_TARGET_HAS_SECURE_ROM_DL_MODE=y
CONFIG_SECURE_SIGNED_APPS_RSA_SCHEME=y
CONFIG_SECURE_BOOT=y
CONFIG_SECURE_BOOT_V2_ENABLED=y
CONFIG_SECURE_BOOT_BUILD_SIGNED_BINARIES=y
CONFIG_SECURE_BOOT_SIGNING_KEY="secure_boot_signing_key.pem"
# CONFIG_SECURE_BOOT_ENABLE_AGGRESSIVE_KEY_REVOKE is not set
CONFIG_SECURE_BOOT_INSECURE=y
CONFIG_SECURE_FLASH_ENC_ENABLED=y
CONFIG_SECURE_FLASH_ENCRYPTION_AES128=y
# CONFIG_SECURE_FLASH_ENCRYPTION_AES256 is not set
# CONFIG_SECURE_FLASH_ENCRYPTION_MODE_DEVELOPMENT is not set
CONFIG_SECURE_FLASH_ENCRYPTION_MODE_RELEASE=y

#
# Potentially insecure options
#
# CONFIG_SECURE_BOOT_ALLOW_JTAG is not set
# CONFIG_SECURE_BOOT_ALLOW_SHORT_APP_PARTITION is not set
# CONFIG_SECURE_BOOT_V2_ALLOW_EFUSE_RD_DIS is not set
# CONFIG_SECURE_BOOT_ALLOW_UNUSED_DIGEST_SLOTS is not set
# end of Potentially insecure options

CONFIG_SECURE_FLASH_CHECK_ENC_EN_IN_APP=y
# CONFIG_SECURE_DISABLE_ROM_DL_MODE is not set
# CONFIG_SECURE_ENABLE_SECURE_ROM_DL_MODE is not set
CONFIG_SECURE_INSECURE_ALLOW_DL_MODE=y
# end of Security features

Build or installation Logs.

No response

More Information.

No response

One issue may be that I am flashing with the following command after burning the encrypted bootloader. Is this uploading a plaintext bootloader and corrupting? idf.py --port /dev/cu.usbserial-110 flash -b 460800

Executing action: flash
Running ninja in directory /xxxxxxxx
Executing "ninja flash"...
[1/5] xxxx
[2/5] Performing build step for 'bootloader'
[1/1] cd xxx/esptool_py && xxx/check_sizes.py --offset 0xa000 bootloader 0x0 xxx/build/bootloader/bootloader.bin
Bootloader binary size 0xa000 bytes. 0x0 bytes (0%) free.
[2/3] cd /xxx/esptool_py/run_serial_tool.cmake
esptool.py esp32s3 -p /dev/cu.usbserial-110 -b 460800 --before=default_reset --after=no_reset --no-stub write_flash --flash_mode dio --flash_freq 80m --flash_size 16MB 0x20000 xxx.bin 0xa000 partition_table/partition-table.bin 0x19000 ota_data_initial.bin
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting....
Chip is ESP32-S3
Features: WiFi, BLE

@davidallenmann

It is not recommended to flash plain-text build artifacts once you enable flash encryption in release mode. I am afraid but this probably means that you have no way to recover this device now. Can you please confirm how you had generated the flash encryption key? Was it through bootloader or you had externally programmed it?

I don't believe I flashed any plain-text build artifacts after enabling flash encryption, unless one of these steps caused a system reset. The three steps below were performed without cycling power or resetting the ESP32-S3

The flash encryption key was generated by the ESP32-S3 I believe using the menuconfig settings.

The secure boot signing key was generated by espsecure.py generate_signing_key.

I performed three flashes usually in this order (on my last attempt I also tried with the secure_certs flash step last instead of first--in case they were getting written over).

secure_certs:

python3 configure_esp_secure_cert.py -p /dev/cu.usbserial-110 --keep_ds_data_on_host --ca-cert root_cert_auth.pem --device-cert $1/cert.pem --private-key $1/private.key --target_chip esp32s3 --secure_cert_type cust_flash
esptool.py --no-stub --port /dev/cu.usbserial-110 write_flash 0xD000 $1/esp_ds_data/esp_secure_cert.bin

encrypted bootloader: esptool.py write_flash using command suggested byidf.py bootloader
binary: Note the command suggested by the idf.py build for flashing generated an error on my second attempt at this. This error is below. So, flashed with:idf.py flash on each of my 3 attempts (I used new modules each time).

I tried this with 3 modules making some changes each time Attempt 1: IDF v4.4 Attempt 2: Updated IDF to latest 4.4.3 Attempt 3: Set to Allow potentially insecure options. And enable ROM download mode. Flash order: Flash bootloader. Flash binary. Flash certs (last).
Note: Viewed serial monitor without board reset and was showing invalid header error

/.espressif/python_env/idf4.4_py3.9_env/bin/python ../../../esp/esp-idf-v4.4.2/components/esptool_py/esptool/esptool.py -p /dev/cu.usbserial-110 -b 460800 --before default_reset --after no_reset --chip esp32s3 --no-stub write_flash --flash_mode dio --flash_size keep --flash_freq 80m 0xa000 build/partition_table/partition-table.bin 0x19000 build/ota_data_initial.bin 0x20000 build/xxxxx.bin
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting....
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:d8:7e:b0
Changing baud rate to 460800
Changed.
Enabling default SPI flash mode...
Configuring flash size...
Flash will be erased from 0x0000a000 to 0x0000afff...
Flash will be erased from 0x00019000 to 0x0001afff...
Flash will be erased from 0x00020000 to 0x00480fff...
Erasing flash...
Took 0.08s to erase flash block
Wrote 3072 bytes at 0x0000a000 in 0.2 seconds (99.9 kbit/s)...
Hash of data verified.
Erasing flash...
Took 0.08s to erase flash block
Wrote 8192 bytes at 0x00019000 in 0.7 seconds (100.1 kbit/s)...
Hash of data verified.
Erasing flash...

A fatal error occurred: Failed to enter Flash download mode (result was 01066430: Operation or feature not supported)

I am using my own boards and programming with a serial programmer that has DTR tied to GPIO0. EN pin is connected to RTS. Not sure if that could cause a reset at some point in the flashing process.

@davidallenmann

Can you please check if you can get the efuse summary of the device using:

espefuse.py --chip esp32s3 summary

If this command does not work then most likely you have secure ROM DL mode enabled on the device, in that case please run following command:

esptool.py --no-stub --chip esp32s3 get_security_info

Please share the results here.

espefuse.py --chip esp32s3 --port  /dev/cu.usbserial-110 summary
espefuse.py v3.3.2
Connecting....

A fatal error occurred: Secure Download Mode is enabled. The tool can not read eFuses.

esptool.py --no-stub --chip esp32s3 --port  /dev/cu.usbserial-110 get_security_info
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting....
Chip is ESP32-S3 in Secure Download Mode
Enabling default SPI flash mode...
Flags: 0x000006f5 (0b11011110101)
Flash_Crypt_Cnt: 0x0
Key_Purposes: (9, 4, 0, 0, 0, 0, 12)
Chip_ID: 9
Api_Version: 0
Hard resetting via RTS pin...

@davidallenmann

Key_Purposes: (9, 4, 0, 0, 0, 0, 12)

As per this both flash enc key and secure boot digest has been programmed on the device. EFuse purpose codes for S3 can be found here.

Flash_Crypt_Cnt: 0x0

This suggests that flash encryption is not yet enabled on this device. Probably flash encryption sequence was not completed on this device. Please try flashing plaintext artifacts (bootloader, partition-table and application) again, and then flash encryption sequence should start again. Note: basic flash writes are supported in secure DL mode with --no-stub argument to esptool.

@ mahavirj Thanks--I will try that.

Do you know why I was encountering this error? I am trying to get this ready for production and ideally will be able to do it as a sequence of flash writes. I should be able to do a series of flashes as described in this document https://docs.espressif.com/projects/esp-idf/en/release-v4.4/esp32s3/security/secure-boot-v2.html

Or--do I need to enable flash encryption as a separate step after enabled secure DL?

I am able to reflash the encyrpted bootloader. I get the following error when flashing the boot table and binary. I got this error the first time I tried writing the binary with esptool.py on one of my previous attempts

python esptool.py -p /dev/cu.usbserial-110 -b 460800 --before default_reset --after no_reset --chip esp32s3 --no-stub write_flash --flash_mode dio --flash_size keep --flash_freq 80m 0xa000 build/partition_table/partition-table.bin 0x19000 build/ota_data_initial.bin 0x20000 build/xxxxxx.bin
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting....
Chip is ESP32-S3 in Secure Download Mode
Changing baud rate to 460800
Changed.
Enabling default SPI flash mode...
Configuring flash size...
Flash will be erased from 0x0000a000 to 0x0000afff...
Flash will be erased from 0x00019000 to 0x0001afff...
Flash will be erased from 0x00020000 to 0x00480fff...
Erasing flash...
Took 0.09s to erase flash block
Wrote 3072 bytes at 0x0000a000 in 0.3 seconds (96.4 kbit/s)...
Erasing flash...
Took 0.13s to erase flash block
Wrote 8192 bytes at 0x00019000 in 0.7 seconds (96.4 kbit/s)...
Erasing flash...

A fatal error occurred: Failed to enter Flash download mode (result was 01060000: Operation or feature not supported)

@davidallenmann

It is difficult to say why this issue occurred in your setup. However, you can enable both flash enc and secure boot simultaneously, there should not be a problem with that.

Regarding flash write error, please see my earlier comment about --no-stub argument to esptool command. Since secure ROM DL mode is already enabled, providing --no-stub is mandatory during flash write. Please refer to docs for this at https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/security/security.html#uart-dl-mode

@mahavirj --no-stub is in the command that fails (see output above). It seems to successfully write the partition table but fails when writing the binary

Aha, my bad! Indeed the command looks correct. I am not sure why it failed when it reached to the application partition erase stage. Would you mind trying this workflow on official S3 devkit once?

I am using my own boards and programming with a serial programmer that has DTR tied to GPIO0. EN pin is connected to RTS. Not sure if that could cause a reset at some point in the flashing process.

Recommend checking this against our official S3 devkits schematics available at https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/index.html#hardware

CC @radimkarnis

@mahavirj I get the same error using the ESP32-S3-WROOM2 DEVKIT-C v1.1 board (changed to octal flash mode in menuconfig and recompiled).

python  esptool.py --chip esp32s3 --port=/dev/cu.usbserial-110 --baud=460800 --before=default_reset --after=no_reset --no-stub write_flash --flash_mode dio --flash_freq 80m --flash_size 16MB 0x0 bootloader.bin               
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting....
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: 68:b6:b3:3b:af:b4
Changing baud rate to 460800
Changed.
Enabling default SPI flash mode...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00009fff...
Erasing flash...
Flash params set to 0x024f
Took 0.29s to erase flash block
Wrote 40960 bytes at 0x00000000 in 3.3 seconds (99.1 kbit/s)...
Hash of data verified.

Leaving...
Staying in bootloader.
python esptool.py -p /dev/cu.usbserial-110 -b 460800 --before default_reset --after no_reset --chip esp32s3 --no-stub write_flash --flash_mode dio --flash_size keep --flash_freq 80m 0xa000 build/partition_table/partition-table.bin 0x20000 build/xxxxx.bin
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting....
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: 68:b6:b3:3b:af:b4
Changing baud rate to 460800
Changed.
Enabling default SPI flash mode...
Configuring flash size...
Flash will be erased from 0x0000a000 to 0x0000afff...
Flash will be erased from 0x00020000 to 0x00480fff...
Erasing flash...
Took 0.08s to erase flash block
Wrote 3072 bytes at 0x0000a000 in 0.2 seconds (101.1 kbit/s)...
Hash of data verified.
Erasing flash...

A fatal error occurred: Failed to enter Flash download mode (result was 01063136: Operation or feature not supported)

python  esptool.py --chip esp32s3 --port=/dev/cu.usbserial-110 --baud=460800 --before=default_reset --after=no_reset --no-stub write_flash --flash_mode dio --flash_freq 80m --flash_size 16MB 0x0 bootloader.bin               
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting....
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: 68:b6:b3:3b:af:b4
Changing baud rate to 460800
Changed.
Enabling default SPI flash mode...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00009fff...
Erasing flash...
Flash params set to 0x024f
Took 0.29s to erase flash block
Wrote 40960 bytes at 0x00000000 in 3.3 seconds (99.1 kbit/s)...
Hash of data verified.

Leaving...
Staying in bootloader.

% python esptool.py -p /dev/cu.usbserial-110 -b 460800 --before default_reset --after no_reset --chip esp32s3 --no-stub write_flash --flash_mode dio --flash_size keep --flash_freq 80m 0xa000 build/partition_table/partition-table.bin 0x20000 build/xxxxx.bin
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting....
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: 68:b6:b3:3b:af:b4
Changing baud rate to 460800
Changed.
Enabling default SPI flash mode...
Configuring flash size...
Flash will be erased from 0x0000a000 to 0x0000afff...
Flash will be erased from 0x00020000 to 0x00480fff...
Erasing flash...
Took 0.08s to erase flash block
Wrote 3072 bytes at 0x0000a000 in 0.2 seconds (101.1 kbit/s)...
Hash of data verified.
Erasing flash...

A fatal error occurred: Failed to enter Flash download mode (result was 01063136: Operation or feature not supported)

Security Info

esptool.py --no-stub --chip esp32s3 --port  /dev/cu.usbserial-110 get_security_info
esptool.py v3.3.2
Serial port /dev/cu.usbserial-110
Connecting...
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: 68:b6:b3:3b:af:b4
Enabling default SPI flash mode...
Flags: 0x00000680 (0b11010000000)
Flash_Crypt_Cnt: 0x7
Key_Purposes: (4, 0, 0, 0, 0, 0, 12)
Chip_ID: 9
Api_Version: 0
Hard resetting via RTS pin...

EFuse Summary (this command works with ESP32-S3-WROOM2

espefuse.py --chip esp32s3 --port  /dev/cu.usbserial-110 summary                   
espefuse.py v3.3.2
Connecting....

=== Run "summary" command ===
EFUSE_NAME (Block) Description  = [Meaningful Value] [Readable/Writeable] (Hex Value)
----------------------------------------------------------------------------------------
Config fuses:
DIS_ICACHE (BLOCK0)                                Disables ICache                                    = False R/W (0b0)
DIS_DCACHE (BLOCK0)                                Disables DCache                                    = False R/W (0b0)
DIS_DOWNLOAD_ICACHE (BLOCK0)                       Disables Icache when SoC is in Download mode       = True R/W (0b1)
DIS_DOWNLOAD_DCACHE (BLOCK0)                       Disables Dcache when SoC is in Download mode       = True R/W (0b1)
DIS_FORCE_DOWNLOAD (BLOCK0)                        Disables forcing chip into Download mode           = False R/W (0b0)
DIS_CAN (BLOCK0)                                   Disables the TWAI Controller hardware              = False R/W (0b0)
DIS_APP_CPU (BLOCK0)                               Disables APP CPU                                   = False R/W (0b0)
FLASH_TPUW (BLOCK0)                                Configures flash startup delay after SoC power-up, = 0 R/W (0x0)
                                                    unit is (ms/2). When the value is 15, delay is 7.
                                                   5 ms                                              
DIS_DIRECT_BOOT (BLOCK0)                           Disables direct boot mode                          = True R/W (0b1)
DIS_USB_SERIAL_JTAG_ROM_PRINT (BLOCK0)             Disables USB-Serial-JTAG ROM printing              = False R/W (0b0)
FLASH_ECC_MODE (BLOCK0)                            Configures the ECC mode for SPI flash             
   = 16-byte to 18-byte mode R/W (0b0)
DIS_USB_SERIAL_JTAG_DOWNLOAD_MODE (BLOCK0)         Disables USB-Serial-JTAG download feature in UART  = False R/W (0b0)
                                                   download boot mode                                
UART_PRINT_CONTROL (BLOCK0)                        Sets the default UART boot message output mode     = Enabled R/W (0b00)
FLASH_TYPE (BLOCK0)                                Selects SPI flash type                             = 8 data lines R/W (0b1)
FLASH_PAGE_SIZE (BLOCK0)                           Sets the size of flash page                        = 0 R/W (0b00)
FLASH_ECC_EN (BLOCK0)                              Enables ECC in Flash boot mode                     = False R/W (0b0)
FORCE_SEND_RESUME (BLOCK0)                         Forces ROM code to send an SPI flash resume comman = False R/W (0b0)
                                                   d during SPI boot                                 
DIS_USB_OTG_DOWNLOAD_MODE (BLOCK0)                 Disables USB-OTG download feature in UART download = False R/W (0b0)
                                                    boot mode                                        
BLOCK_USR_DATA (BLOCK3)                            User data                                         
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 

Efuse fuses:
WR_DIS (BLOCK0)                                    Disables programming of individual eFuses          = 8388864 R/W (0x00800100)
RD_DIS (BLOCK0)                                    Disables software reading from BLOCK4-10           = 1 R/W (0b0000001)

Identity fuses:
SECURE_VERSION (BLOCK0)                            Secure version (used by ESP-IDF anti-rollback feat = 0 R/W (0x0000)
                                                   ure)                                              
MAC (BLOCK1)                                       Factory MAC Address                               
   = 68:b6:b3:3b:af:b4 (OK) R/W 
WAFER_VERSION (BLOCK1)                             WAFER version                                      = 1 R/W (0b001)
PKG_VERSION (BLOCK1)                               ??? Package version                                = ESP32-S3 R/W (0x0)
BLOCK1_VERSION (BLOCK1)                            ??? BLOCK1 efuse version                           = 1 R/W (0b001)
OPTIONAL_UNIQUE_ID (BLOCK2)                        ??? Optional unique 128-bit ID                    
   = b9 6b 9f 39 3c 93 b0 25 09 87 11 ac 86 48 db 10 R/W 
BLOCK2_VERSION (BLOCK2)                            ??? Version of BLOCK2                              = 4 R/W (0b100)
CUSTOM_MAC (BLOCK3)                                Custom MAC Address                                
   = 00:00:00:00:00:00 (OK) R/W 

Security fuses:
SOFT_DIS_JTAG (BLOCK0)                             Software disables JTAG by programming odd number o = 0 R/W (0b000)
                                                   f 1 bit(s). JTAG can be re-enabled via HMAC periph
                                                   eral                                              
HARD_DIS_JTAG (BLOCK0)                             Hardware disables JTAG permanently                 = True R/W (0b1)
DIS_DOWNLOAD_MANUAL_ENCRYPT (BLOCK0)               Disables flash encryption when in download boot mo = False R/W (0b0)
                                                   des                                               
SPI_BOOT_CRYPT_CNT (BLOCK0)                        Enables encryption and decryption, when an SPI boo = Enable R/W (0b111)
                                                   t mode is set. Enabled when 1 or 3 bits are set,di
                                                   sabled otherwise                                  
SECURE_BOOT_KEY_REVOKE0 (BLOCK0)                   Revokes use of secure boot key digest 0            = False R/W (0b0)
SECURE_BOOT_KEY_REVOKE1 (BLOCK0)                   Revokes use of secure boot key digest 1            = False R/W (0b0)
SECURE_BOOT_KEY_REVOKE2 (BLOCK0)                   Revokes use of secure boot key digest 2            = False R/W (0b0)
KEY_PURPOSE_0 (BLOCK0)                             KEY0 purpose                                       = XTS_AES_128_KEY R/- (0x4)
KEY_PURPOSE_1 (BLOCK0)                             KEY1 purpose                                       = USER R/W (0x0)
KEY_PURPOSE_2 (BLOCK0)                             KEY2 purpose                                       = USER R/W (0x0)
KEY_PURPOSE_3 (BLOCK0)                             KEY3 purpose                                       = USER R/W (0x0)
KEY_PURPOSE_4 (BLOCK0)                             KEY4 purpose                                       = USER R/W (0x0)
KEY_PURPOSE_5 (BLOCK0)                             KEY5 purpose                                       = USER R/W (0x0)
SECURE_BOOT_EN (BLOCK0)                            Enables secure boot                                = False R/W (0b0)
SECURE_BOOT_AGGRESSIVE_REVOKE (BLOCK0)             Enables aggressive secure boot key revocation mode = False R/W (0b0)
STRAP_JTAG_SEL (BLOCK0)                            Enable selection between usb_to_jtagor pad_to_jtag = False R/W (0b0)
                                                    through GPIO3                                    
DIS_DOWNLOAD_MODE (BLOCK0)                         Disables all Download boot modes                   = False R/W (0b0)
ENABLE_SECURITY_DOWNLOAD (BLOCK0)                  Enables secure UART download mode (read/write flas = False R/W (0b0)
                                                   h only)                                           
BLOCK_KEY0 (BLOCK4)
  Purpose: XTS_AES_128_KEY
    Encryption key0 or user data                      
   = ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? -/- 
BLOCK_KEY1 (BLOCK5)
  Purpose: USER
               Encryption key1 or user data                      
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 
BLOCK_KEY2 (BLOCK6)
  Purpose: USER
               Encryption key2 or user data                      
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 
BLOCK_KEY3 (BLOCK7)
  Purpose: USER
               Encryption key3 or user data                      
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 
BLOCK_KEY4 (BLOCK8)
  Purpose: USER
               Encryption key4 or user data                      
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 
BLOCK_KEY5 (BLOCK9)
  Purpose: USER
               Encryption key5 or user data                      
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 
BLOCK_SYS_DATA2 (BLOCK10)                          System data (part 2)                              
   = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 R/W 

Spi_Pad_Config fuses:
SPI_PAD_CONFIG_CLK (BLOCK1)                        SPI CLK pad                                        = 0 R/W (0b000000)
SPI_PAD_CONFIG_Q (BLOCK1)                          SPI Q (D1) pad                                     = 0 R/W (0b000000)
SPI_PAD_CONFIG_D (BLOCK1)                          SPI D (D0) pad                                     = 0 R/W (0b000000)
SPI_PAD_CONFIG_CS (BLOCK1)                         SPI CS pad                                         = 0 R/W (0b000000)
SPI_PAD_CONFIG_HD (BLOCK1)                         SPI HD (D3) pad                                    = 0 R/W (0b000000)
SPI_PAD_CONFIG_WP (BLOCK1)                         SPI WP (D2) pad                                    = 0 R/W (0b000000)
SPI_PAD_CONFIG_DQS (BLOCK1)                        SPI DQS pad                                        = 0 R/W (0b000000)
SPI_PAD_CONFIG_D4 (BLOCK1)                         SPI D4 pad                                         = 0 R/W (0b000000)
SPI_PAD_CONFIG_D5 (BLOCK1)                         SPI D5 pad                                         = 0 R/W (0b000000)
SPI_PAD_CONFIG_D6 (BLOCK1)                         SPI D6 pad                                         = 0 R/W (0b000000)
SPI_PAD_CONFIG_D7 (BLOCK1)                         SPI D7 pad                                         = 0 R/W (0b000000)

Usb Config fuses:
DIS_USB (BLOCK0)                                   Disables the USB OTG hardware                      = False R/W (0b0)
USB_EXCHG_PINS (BLOCK0)                            Exchanges USB D+ and D- pins                       = False R/W (0b0)
EXT_PHY_ENABLE (BLOCK0)                            Enables external USB PHY                           = False R/W (0b0)
BTLC_GPIO_ENABLE (BLOCK0)                          Enables BTLC GPIO                                  = 0 R/W (0b00)
DIS_USB_JTAG (BLOCK0)                              Disable usb_serial_jtag-to-jtag function           = True R/W (0b1)
DIS_USB_SERIAL_JTAG (BLOCK0)                       Disable usb_serial_jtag module                     = False R/W (0b0)
USB_PHY_SEL (BLOCK0)                               Select internal/external PHY for USB OTGand usb_se = False R/W (0b0)
                                                   rial_jtag                                         

Vdd_Spi Config fuses:
VDD_SPI_XPD (BLOCK0)                               The VDD_SPI regulator is powered on                = True R/W (0b1)
VDD_SPI_TIEH (BLOCK0)                              The VDD_SPI power supply voltage at reset          = Connect to 1.8V LDO R/W (0b0)
VDD_SPI_FORCE (BLOCK0)                             Force using VDD_SPI_XPD and VDD_SPI_TIEH to config = True R/W (0b1)
                                                   ure VDD_SPI LDO                                   
PIN_POWER_SELECTION (BLOCK0)                       Sets default power supply for GPIO33..37           = VDD_SPI R/W (0b1)

Wdt Config fuses:
WDT_DELAY_SEL (BLOCK0)                             Selects RTC WDT timeout threshold at startup       = 0 R/W (0b00)

Flash voltage (VDD_SPI) set to 1.8V by efuse.

I also tried increasing the partition table offset just in case there needed to be space (as it was there was exact byte alignment)

A test on a new board works with only Flash encryption enabled and UART ROM download mode permanently disabled. So, it seems to have something to do with Secure Boot.

Hi @davidallenmann! I tested SB + FE for ESP32-S3 on the IDF v4.4.3-347-g9ee3c8337d version, it works, no issue that you have faced.

Here https://github.com/espressif/esp-idf/issues/10535#issuecomment-1385642447, it is strange why do you flash the bootloader twice?

Could you describe all steps that you do? I do not fully understand it.

In a simple way you need:

have erased flash
idf.py build with enabled options:
- CONFIG_SECURE_BOOT=y
- CONFIG_SECURE_BOOT_V2_ENABLED=y
- CONFIG_SECURE_BOOT_SIGNING_KEY="test/secure_boot_signing_key.pem"
- CONFIG_SECURE_FLASH_ENC_ENABLED=y
idf.py bootloader-flash (flash only bootloader). SB and FE will not be enabled until the partition table and app be found.
idf.py flash monitor. If you have an issue at this stage then you need to flash the partition table at the end. It will help you to prevent enabling FE and SB until everything will be completely flashed.

What is your connection to chip? Is it a built-in USB-UART into S3 chip or an external USB-UART?

Minor: about Partition Table. You need to remove all offsets in your partition_table.csv file and use only CONFIG_PARTITION_TABLE_OFFSET=0xC000, in this case, esp_secure_cert should go into 0xD000 and the following go right after the previous partition. You will get the same result, and the ability to move it, only change one Kconfig option (CONFIG_PARTITION_TABLE_OFFSET).

The following worked. I think the issue may have either been the order of operations or the partition table offset being wrong. Thanks so much for your help.

Using ESP32-S3-WROOM2 DevKit-C and USB cable connected to UART port.

Set partition table offset to 0xC000 and removed offsets in partition_table.csv

esptool.py --port /dev/cu.usbserial-110 erase_flash
idf.py bootloader
python esptool.py --chip esp32s3 --port=/dev/cu.usbserial-110 --baud=460800 --before=default_reset --after=no_reset --no-stub write_flash --flash_mode dout --flash_freq 80m --flash_size 16MB 0x0 bootloader.bin
Flash security creds
idf.py build
idf.py --port /dev/cu.usbserial-110 flash -b 1500000 monitor

Summary: The bootloader with SB + FE for ESP32-S3 takes 45056 bytes [0x0000 - 0xAFFFF]. (what is your bootloader size?) Your partition table was at offset 0xA000 (see CONFIG_PARTITION_TABLE_OFFSET in sdkconfig) So it means that the last sector of the bootloader was overwritten when you flash the partition table. The last sector of the bootloader is the signature of the secure boot.

The build system should throw an error.

Error: Bootloader binary size 0xb000 bytes is too large for partition table offset 0xa000. Bootloader binary can be maximum 0xa000 (40960) bytes unless the partition table offset is increased in the Partition Table section of the project configuration menu.

But it is still not clear to me how you get invalid header: 0x507bc083, seems, at some point, you overwrite the bootloader with a new one after flash encryption was already enabled. Please expose your steps how did you reach this state? (it can help us to prevent it in the future for other users). Thanks.

Still working on this. I need to build some more boards for testing.

My last flashing attempt produced a new error. It does seem that the bootloader gets built again after running idf.py flash, but it doesn't appear to be flashed again


--- idf_monitor on /dev/cu.usbserial-110 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x28 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DOUT, clock div:1
load:0x3fce3970,len:0x2664
load:0x403c9700,len:0xcc4
load:0x403cc700,len:0x4dcc
SHA-256 comparison failed:
Calculated: 358f2d2d1df04ede038c399874767513cb11e9e5e4c9e74056f873e45d28a274
Expected: 04aafc516eb8044f78d68a86c37026f50ca3487a3b6b1ad83f904082ad928a45
Attempting to boot anyway...
entry 0x403c9928
Sig block 0 invalid: Image digest does not match
E (956) secure_boot_v2: Secure Boot V2 verification failed.
E (956) esp_image: Secure boot signature verification failed
E (966) esp_image: Image hash failed - image is corrupt
E (968) secure_boot_v2: bootloader image appears invalid! error 8194
E (975) boot: Secure Boot v2 failed (8194)
E (979) boot: Factory app partition is not bootable
E (985) esp_image: image at 0x520000 has invalid magic byte (nothing flashed here?)
E (993) boot: OTA app partition slot 0 is not bootable
E (999) esp_image: image at 0xa20000 has invalid magic byte (nothing flashed here?)
E (1007) boot: OTA app partition slot 1 is not bootable
E (1013) boot: No bootable app partitions in the partition table

One potential issue is that I am using a legacy format for secure cert and did not Enable DS peripheral support in ESP Secure Cert Manager. I also did not update secure cert library (v2.0.2) when I updated IDF to v4.4.3-347-g9ee3c8337d

I am updating secure cert library, enabling DS peripheral, and updating to use --secure_cert_type cust_flash_tlv

I have been flashing secure certs from a bash script with (which does not contain --efuse_key_id 1):

python3 configure_esp_secure_cert.py -p /dev/cu.usbserial-110 --keep_ds_data_on_host --ca-cert root_cert_auth.pem --device-cert $1/cert.pem --private-key $1/private.key --target_chip esp32s3 --secure_cert_type cust_flash

esptool.py --no-stub --port /dev/cu.usbserial-110 write_flash 0xD000 $1/esp_ds_data/esp_secure_cert.bin

Updating to latest secure cert library still gives the no bootable app partitions issue.

I first uploaded without Secure Boot enabled and without flash encryption to make sure the new secure cert library was working correctly, which it was.
Turned on Secure Boot and Flash Encryption.
Built and flashed bootloader
idf.py build
idf.py flash monitor

SDKCONFIG

#
# Security features
#
CONFIG_SECURE_BOOT_SUPPORTS_RSA=y
CONFIG_SECURE_TARGET_HAS_SECURE_ROM_DL_MODE=y
# CONFIG_SECURE_SIGNED_APPS_NO_SECURE_BOOT is not set
# CONFIG_SECURE_BOOT is not set
CONFIG_SECURE_FLASH_ENC_ENABLED=y
CONFIG_SECURE_FLASH_ENCRYPTION_AES128=y
# CONFIG_SECURE_FLASH_ENCRYPTION_AES256 is not set
CONFIG_SECURE_FLASH_ENCRYPTION_MODE_DEVELOPMENT=y
# CONFIG_SECURE_FLASH_ENCRYPTION_MODE_RELEASE is not set

#
# Potentially insecure options
#
# CONFIG_SECURE_BOOT_ALLOW_JTAG is not set
CONFIG_SECURE_FLASH_UART_BOOTLOADER_ALLOW_ENC=y
# CONFIG_SECURE_FLASH_UART_BOOTLOADER_ALLOW_CACHE is not set
# CONFIG_SECURE_FLASH_REQUIRE_ALREADY_ENABLED is not set
# end of Potentially insecure options

CONFIG_SECURE_FLASH_CHECK_ENC_EN_IN_APP=y
# CONFIG_SECURE_DISABLE_ROM_DL_MODE is not set
# CONFIG_SECURE_ENABLE_SECURE_ROM_DL_MODE is not set
CONFIG_SECURE_INSECURE_ALLOW_DL_MODE=y
# end of Security features

Hi @davidallenmann!

You get the issue SHA-256 comparison failed because your settings in sdkconfig and esptool.py write_flash ....bootloader.bin are different. The bootloader image is overwritten by the given options (--flash_mode dout --flash_freq 80m --flash_size 16MB) during writing (note that, the bootloader file is not changed). Please make sure that ESPTOOLPY_FLASHMODE ESPTOOLPY_FLASHFREQ ESPTOOLPY_FLASHSIZE are set as required.

This log is from ROM bootloader (it says that the 2nd stage bootloader image was changed):

SHA-256 comparison failed:
Calculated: 75267704ce9a3f91b5be85dcf5669a015759d3683ff7e3d6d86e11124b13aa70
Expected: f1354bfaeb76fdbdf87fcce2accbb10a850ba5ae76ada3661fba9e7a86a62a5b

When your sdkconfig options are not consistent with --flash_... options then the esptool overwrites the header of the bootloader but does not update the sha256 at the end of the image and signatures. So it makes secure boot not workable.

python  esptool.py --chip esp32s3 --port=/dev/cu.usbserial-110 --baud=460800 \
--before=default_reset  --after=no_reset --no-stub \
write_flash \
--flash_mode dout \
--flash_freq 80m \
--flash_size 16MB \
0x0 bootloader.bin

When the image flash configs are overwritten then you see this log: Flash params set to 0x034f

Flashing bootloader with mode dio fixed the issue. I have not been able to recreate my original issue, but potentially updating libraries fixed the problem. I'm ok with closing this.

I am glad to hear that you managed to get it workable. (I suspect that the original issue was that the bootloader was overwritten by the plain-text image when FE was already enabled.)

espressif / esp-idf