[question] Erasing Flash efficiently when using an RTOS

[Dave-Nocturnal]

I am starting a project and plan to use LittleFS. We will have FreeRTOS running and use littleFS mostly for logging.

The problem I see with logging in file systems is task blocking, mostly during page erasure. I don't understand the details of when FreeRTOS decides to erase a page so if there is a good description of this strategy, please point me at it.

I guess I should ask, does LittleFS support multithreading? What do I need to do for this? I would not be writing to the same file concurrently, but there may be more than one open file handle in different tasks.

In my mind, it would be ideal if the used blocks could be put into a queue and erased slowly, to be returned to LittleFS when they have been scrubbed and are ready for use. There are enough spare blocks in external flash to tolerate a slow return cycle and I am trying to minimize blocking for application tasks that use logging. Is it possible to do something like this with LittleFS, or are there other tricks/considerations when using FreeRTOS?

I would think LittleFS has some method for figuring out if a file system is partially dirty and when pages need to be erased, so that if we reset during erasure there will be no lost or unusable Flash pages. It might take a bit longer to clean up the managed storage area but it will still work as expected. Is that correct?

[e107steved]

Have a dedicated thread to manage the logging, with a queue of events to log. Then (provided the queue doesn't overflow) you've isolated littleFS from the rest of the application. If you might call littleFS from other threads, you'll need to manage that; littleFs has hooks for a mutex, or you can implement your own mutex round calls to littleFS functions.

Incidentally, littleFS is good for "growing" logs, where you just append entries; can be rather slow if implementing a circular log.

[sslupsky]

Incidentally, littleFS is good for "growing" logs, where you just append entries; can be rather slow if implementing a circular log.

You mention it is good for growing logs. I have experienced issues where when the file grows larger than a few 10's of KB and I seek the end of the file, it can take several seconds for LFS return. During this time, LFS doesn't yield so I have experienced this delay can lead to issues where a watchdog may timeout.

[Dave-Nocturnal]

Have a dedicated thread to manage the logging, with a queue of events to log.

Yes, I understand the workarounds. I was hoping for something a bit more elegant. Throwing RAM buffers at the situation is not always easy. If pages needing erasure could be queued, scrubbed, and returned in a non-blocking approach that doesn't include RAM buffering, that would be ideal.

I have not found a good explanation of how wear leveling works in lfs, is there a place that explains how this happens?

[GaryJSAdv]

I'm also interested in an offline garbage collection that I could run in an idle task. I've got a circular log under LittleFS - it's working although performance could be better. @Dave-Nocturnal - Any use: http://littlefs.geky.net/demo.html