espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.42k stars 7.25k forks source link

Pls increase NVS key size to store UUID (IDFGH-13613) #14498

Open baiomys opened 3 weeks ago

baiomys commented 3 weeks ago

Is your feature request related to a problem?

No response

Describe the solution you'd like.

NVS storage can be organized to store JSON data in relational database style via linked-list, but to allow direct access without iterator, key should hold some distinct ID like UUID or MD5 hash.

Describe alternatives you've considered.

https://github.com/armink/FlashDB

Additional context.

No response

rrtandler commented 2 weeks ago

Hi @baiomys ,

The primary focus of NVS was to introduce the simple key value store for configuration data with a focus to the reliability and operation consistency. All there with respect to the limited nature of embedded environment and aim for minimal memory footprint. The key size is a part of the physical layout of the metadata storage unit called "entry". At the moment the metadata of the entry is using all the space available, so there is no easy way to introduce longer keys with reasonable amount of efforts. We do not plan to extend the NVS this way.

You have outlined the way to achieve the direct access to your data. Calculate 16 byte hash, save complete UUID under this hash (into namespace holding hash to uuid relation) and then store JSON under the same hash into another namespace holding the hash to JSON relation. To handle potential hash collision scenarios, you have to introduce some mechanism distinguishing the colliding UUIDS. I.e. introduce the hash to address colliding multiple UUIDs record. There store the list of all UUIDs to intermediate (i.e. simple sequential number) keys to uniquely address the particular JSON. And finally store the JSON using the intermediate key.