NVS from un-encrypted to encrypted Flash (IDFGH-12566)

[mgoeller]

Answers checklist.

• [X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• [X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• [X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

I want to migrate my currently installed ESP32-S3 based devices from un-encrypted flash to encrypted flash. It is not feasible to do this manually, I need the procedure to happen in a single OTA update process.

What I achieved in tests so far:

• working flash encryption in general
• bootloader update

Problem: NVS partition with existing custom data needs to be migrated as well. Once the NVS partition is initialised encrypted it cannot read the previously stored data (un-encrypted) anymore. What is the best process for this?

[0xFEEDC0DE64]

you cannot update the partition table remotely, hence it will be hard to add the nvs keys partition, also flash encryption must be enabled for bootloader too, if I'm not mistaken.

[mgoeller]

We already implemented the OTA update for partition table and bootloader. That works fine. The only missing is the migration of the NVS partition, in particular, its content.

[AdityaHPatwardhan]

Hi @mgoeller, Update an NVS partition from non-encrypted state to encrypted state via OTA is not supported. As if you need to encrypt the NVS partition, you need to read all the entries and then write them in another NVS partition which has been initialised with nvs_secure_flash_init_partition() ( which initialises the partition as encrypted).

So I think the workflow to update the encrypted partition from non-encrypted would be as follows: 1) Update the partition table to add one more nvs_partition and one nvs_keys partition.

We shall first store the NVS encrypted data in this new partition and then use this new partition as the default NVS partition by updating the partition table.

2) There shall be 2 OTA, first OTA shall add a new NVS partition, initialise it with nvs_secure_flash_init_partition() ( which initialises the partition as encrypted) and then copy all the contents from the current NVS partition into the new NVS partition. For copying the contents from non-encrypted to encrypted NVS partition, the API nvs_entry_next could be used

The second OTA shall update the partition table once again to make this new NVS partition as the default one and erase the non-encrypted one. (Could also be done in a single OTA but I think it would be less reliable.)

[mgoeller]

It seems like it's possible to encrypt the NVS partitions with the same key as the rest of the flash. Why is this not advised / no-where mentioned by Espressif? See here: https://www.esp32.com/viewtopic.php?t=28715

[AdityaHPatwardhan]

Hi @mgoeller, I checked the forum post. Technically it may be possible to use flash encryption key directly for NVS encryption but it is not as straightforward as it seems. As explained in the thread NVS contains some data that must not be encrypted and some data that needs to be encrypted. It would require following conditions for it to work. 1) Flash encryption is enabled on the device 2) While enabling flash encryption the data in NVS which must not be encrypted is kept as it is kept as plaintext and only other data is encrypted or NVS partition is completely empty. 3) NVS partition is marked as encrypted so esp_partition* APIs directly utilise flash encryption key for writing and reading the data.

However this is not directly supported in esp-idf. In esp-idf when flash encryption is enabled through bootloader, the bootloader encrypts all the partitions which are marked as encrypted. So if an NVS partition exists then we somehow need to tell the bootloader to only encrypt the necessary data and keep the data that must not be encrypted as plaintext. This part is not currently present in the bootloader. This part may add additional complexity in the bootloader which may not be required at the moment.

Additionally this would cause an issue when one wants to generate the encrypted NVS partition outside of the device. For this they would need to use the Flash Encryption key outside of the chip which is not recommended. Usually it is generated internally and then stored with R/W protection enabled.

As this deviates from the standard workflow that are in place for NVS. It has not integrated in esp-idf at the moment.

For your case, the NVS partition already contains the data. If you plan to use the FE key for NVS then following things would change 1) You would need to first read the plaintext NVS and then write then write that data into a new NVS partition which has been marked as encrypted. This way the non-encrypted data should be handled internally by NVS 2) Make the new nvs partition as the default NVS through updating the partition table. 3) This would remove the requirement of nvs_encryption keys and thus save 1 flash sector size of 4K.

But as I see this is pretty much the same compared to the above just with the addition of a new key. So I would recommend using the standard NVS encryption mechanism.

CC @igrr

[AdityaHPatwardhan]

Hi @mgoeller Have you had time to go through the suggestions?

[AdityaHPatwardhan]

Hi @mgoeller, Closing this issue since the question seems to be answered, please feel free to re-open if required. Thanks, aditya