search

tweedegolf / sequential-storage

A crate for storing data in flash memory with minimal need for erasing pages
Apache License 2.0
87 stars 8 forks source link

Requirements for using sequential-storage #30

Closed HaoboGu closed 4 months ago

HaoboGu commented 4 months ago

Hi there, I'm searching for a efficient KV store for embedded devices. sequential-storage looks very great, but I have several questions about the library:

  1. I have only 2 sectors(16KB) available for storing data in my MCU's flash. Is it enough for using sequential-storage? Can I use only 1 sector?

  2. I found that sequential-storage requires embedded-storage-async, but my HAL implements only embedded-storage, does it mean that I cannot use sequential-storage for my case?

  3. I'm going to use map, I saw that the API requires only a range of Flash, is there any limitation of flash range? Can I pass different range for different data, or make the range start/end at the middle of a flash sector?

diondokter commented 4 months ago

Hey there!

Question 1 and 3 are answered by these asserts: https://github.com/tweedegolf/sequential-storage/blob/067b99d81451c73160ed2d9609304a0a4b6ea79e/src/map.rs#L175-L177

You need at least two pages and the range has to fall on the page boundaries. The amount of storage you'll get with your two pages is 8KB. One page is left empty until the other page is fully filled up. At that point all the active data of the active page is migrated to the empty page and the active page is erased to make it empty again. This is the only way to guarantee no data loss.

The queue implementation can use one page because it doesn't do any migration of data.

Question 2: Up until version 0.6 this crate was fully blocking and used the non-async variant. But depending on your flash and system performance, it could be very slow (100's of ms) and having all that time blocking is not very nice. So this crate was converted to async. But don't worry! While there is no way to make something blocking be async, there is a way to make something async be blocking.

If you wrap this wrapper around your flash, then it will implement the async trait based on the normal trait. This will still actually block with all the downsides of that, but to Rust it'll seem like it's async so you can use it.

You'll still have to run the async functions somehow. If you don't have an executor you could use some simple block_on executor like this one or this one.

Hope this helps!

HaoboGu commented 4 months ago

@diondokter Thanks a lot! I'll try this lib in my project :D