When Flash Encryption is enabled, can not write the same 16 byte block with diferent values (IDFGH-13984)

carlessole commented 3 weeks ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.3

Espressif SoC revision.

ESP32-WROVER-E-N16R8

Operating System used.

Windows

How did you build your project?

Eclipse IDE

If you are using Windows, please specify command line type.

None

Development Kit.

Custom Board

Power Supply used.

USB

What is the expected behavior?

If Flash Encryption is enabled, when running following code:

Situation 1)

// Coming from xFlash region completely erased

uint8_t data_write[16];
uint8_t data_read[16];
for (int i = 0; i < 16; i++)
    data_write[i] = i;

// 1)
esp_partition_write(storage_partition, addr, data_write, 16);
esp_partition_read(storage_partition, addr, data_read, 16);

// 2)
esp_partition_write(storage_partition, addr, data_write, 16);
esp_partition_read(storage_partition, addr, data_read, 16);

// 3)
data_write[6] = 0x25;
esp_partition_write(storage_partition, addr, data_write, 16);
esp_partition_read(storage_partition, addr, data_read, 16);

I was expecting the same behavior as when Flash Encryption is NOT enabled. This means, that after step 3, comparing data_write and data_read, just the position 6 is different.

Situation 2) [Not fully tested, I will do it on Monday)

// Coming from xFlash region completely erased

uint8_t data_write[16];
uint8_t data_read[16];

esp_partition_read(storage_partition, addr, data_read, 16);
esp_partition_read(storage_partition, addr, data_write, 16);

for (int i = 0; i < 16; i++)
{
    if(i != 6)
        data_write[i] = i;
}

// 1)
esp_partition_write(storage_partition, addr, data_write, 16);
esp_partition_read(storage_partition, addr, data_read, 16);

// 2)
esp_partition_write(storage_partition, addr, data_write, 16);
esp_partition_read(storage_partition, addr, data_read, 16);

// 3)
data_write[6] = 0x25;
esp_partition_write(storage_partition, addr, data_write, 16);
esp_partition_read(storage_partition, addr, data_read, 16);

I was expecting the same behavior as when Flash Encryption is NOT enabled. This means, that after step 3, comparing data_write and data_read all the data is the same

What is the actual behavior?

Situation 1) 1) data_write and data_read have the same value 2) data_write and data_read have the same value 3) data_write and data_read have completely different values on all its elements, not just on position 6.

Situation 2) (Still pending to check it 100%) 1) data_write and data_read have the same value 2) data_write and data_read have the same value 3) data_write and data_read have completely different values on all its elements.

Steps to reproduce.

See What is the expected behavior? section

Debug Logs.

No response

More Information.

It is probably a consideration when handling with Flash Encryption as it works with 16 byte blocks encryption. But, is there a way to be able to have the same behavior as the one when Flash Encryption is NOT enabled.

igrr commented 3 weeks ago

It is probably a consideration when handling with Flash Encryption as it works with 16 byte blocks encryption. But, is there a way to be able to have the same behavior as the one when Flash Encryption is NOT enabled.

That's right, when flash encryption is used, the whole block length (16 bytes) is written at once. Changing the plaintext by one byte completely changes the ciphertext, so we can no longer rely on this common behavior of NOR flash.

Quoting the docs:

Partitions marked with an encryption flag will automatically be written via the esp_flash_write_encrypted() function. If writing to an encrypted partition, all write offsets and lengths must be multiples of 16 bytes. See the esp_flash_write_encrypted() function for more details. Unencrypted partitions do not have this restriction.

This behavior can't be "hidden" by Flash driver, you need to take it into account in your application.

Some examples of how we deal with it:

in NVS, key-value pair occupies 32 bytes, so it can be written at once. Longer strings and blobs are split into 32-byte chunks. Entry state bitmap (2 bits per entry) is stored without flash encryption (using esp_partition_write_raw).
in wear_levelling component, we write the whole 16-byte blocks each time we update the internal counter. In other words, 1 increment of the counter is stored as a 16-byte block.
in FATFS, writes are always done in sector sized chunks (512 or 4096 bytes), so 16-byte encryption blocks are not an issue
SPIFFS filesystem design expects the same NOR flash writing behavior as you expect, so SPIFFS can only be used in unencrypted partitions

If you meet trouble adapting your application to this behavior, please describe what you would like to achieve and we'll try to help you.

carlessole commented 3 weeks ago

Thanks @igrr, I understand perfectly. Now what we think has been confirmed.

We have "our own" 'key-value' logic implemented in code shared between different platforms/microcontrollers and when running it under a Flash Encrypted ESP32 board, the problem arises. We will redesign a little bit to be able to run properly also under these circumstances.

I will take a look to the documentation and try to apply it to our existing code.

Thanks

espressif / esp-idf