NVS runs out of space before expected. (IDFGH-11359)

amrsoll commented 8 months ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

b4268c874a4cf8fcf7c0c4153cffb76ad2ddda4e

ESP-IDF v5.2-dev-3775-gb4268c874a

Espressif SoC revision.

ESP32-S3

Operating System used.

Linux

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

Development Kit.

ESP32 S3 mini

Power Supply used.

USB

What is the expected behavior?

I try saving multiple blobs to the NVS. The NVS partition is 16KB, and the blob sizes are 6 KB and 6 KB respectively.

What is the actual behavior?

The NVS dos not have enough memory.

Steps to reproduce.

git clone git@github.com:amrsoll/nvs_rw_blobs.git
cd nvs_rw_blobs
idf.py flash monitor

Debug Logs.

I (341) main_task: Started on CPU0
I (351) main_task: Calling app_main()
Nothing saved yet!
I (541) storage: Saving blob, pass 1
Writing : 6000
Writing : 6000
W (601) storage: Error (ESP_ERR_NVS_NOT_ENOUGH_SPACE) saving run time blob to NVS!
Blob size : 6000
I (721) storage: Saving blob, pass 2
Writing : 6000
Writing : 6000
W (1031) storage: Error (ESP_ERR_NVS_NOT_ENOUGH_SPACE) saving run time blob to NVS!
I (1141) storage: Saving blob, pass 3
E (1141) storage: Error (ESP_FAIL) reading data from NVS!
Writing : 6000
Writing : 6000
W (1251) storage: Error (ESP_ERR_NVS_NOT_ENOUGH_SPACE) saving run time blob to NVS!
I (1361) storage: Saving blob, pass 4
E (1361) storage: Error (ESP_FAIL) reading data from NVS!

More Information.

See my repository for the source files to reproduce error

https://github.com/amrsoll/nvs_rw_blobs

amrsoll commented 8 months ago

I fixed a memory leak in the original example.

However, the NVS partition does not want to allocate space for 2x 6KB blobs. I updated the example

AxelLin commented 7 months ago

 * // Example of nvs_get_stats() to get the number of used entries and free entries:
 * nvs_stats_t nvs_stats;
 * nvs_get_stats(NULL, &nvs_stats);
 * printf("Count: UsedEntries = (%d), FreeEntries = (%d), AllEntries = (%d)\n",
          nvs_stats.used_entries, nvs_stats.free_entries, nvs_stats.total_entries);

amrsoll commented 7 months ago

So you're saying I should check for available space before committing? How does that explain the bahaviour I am seeing?

rrtandler commented 7 months ago

Hi @amrsoll,

Please look at the documentation of nvs_stats_t returned by nvs_get_stats. This structure was extended by the member available_entries (not shown in @AxelLin 's example) which indicates useable space for data. Please note, that (over)writing blobs requires extra entries for blob data (one entry per 32 bytes of data) and 2 entries for overhead data (index and key). If you are overwriting existing blob (re-using same key), the data occupied by the old value will be returned to the pool of available entries first after successful writing of the new value. It effectively means, that during the write operation, you will need space for both old and new versions (data as well as overhead).

espressif / esp-idf