how to use littlefs with Freertos

[NewbieYangbo]

Hi, I have an experiment project. Run FreeRtos + Littlefs + Nordic SDK on NRF52840. I have experience with Littlefs + Nordic SDK on NRF52832. And the file system works well. Project of FreeRtos/Nordic SDK on NRF52840 is usable. I just replaced malloc/free with pvPortMalloc/vPortFree(p) in Littlefs, But I can not mount the file system with wrong -84. It seems i missed something to mount file system under Freertos. I am a newbie about this. Is there any bro give me some examples.

[maboytim]

FreeRTOS really has almost nothing to do with it. The only place FreeRTOS would be involved is if you are using the LFS_THREADSAFE option where you would uses a freertos mutex to serialize certain lfs operations - but for simple testing with lfs access from only a single task that isn't necessary.

However, since you are using the freertos heap, freertos supports multiple heap models - you probably should use heap 3, 4, or 5.

But it is likely an issue with your lfs configuration or your underlying block device driver.

Note the lfs filesystem needs to be formatted before it can be mounted.

If you can, single step into the mount operation to see what lfs doesn't like - a lot of things could result in a -84.

[desmond-lim-tfs]

Hi, I am having the similar problem too using FreeRTOS. Occasionally, I can mount little FS after trying to mount several times. However, most of the time I failed to mount it as what was being mentioned above. Even the formatting is not stable as well with more failure.

From the discussion above, I am using heap 4, and I don't think I use any LFS_THREADSAFE option at all. Should I enable the LFS_THREADSAFE options?

The lfs configuration is as follows which works in non-FreeRTOS environment.

const struct lfs_config LittleFS_config = { .context = (void*)&LittleFS_ctx, .read = lfs_mflash_read, .prog = lfs_mflash_prog, .erase = lfs_mflash_erase, .sync = lfs_mflash_sync, .read_size = 16, .prog_size = 256, .block_size = 4096, .block_count = 1024, .cache_size = 256, .block_cycles = 200, .lookahead_size = 16 };

And the underlying block device driver has been tested working fine.

I follow the examples where I try to mount before format, and if it failed, I will format then remount it. Yet same thing happened. And I tried to single step through it but when it hit the lfs_raw, some how, it just return zero,,

Hope you are able to help on this. Thanks.

[maboytim]

I use lfs successfully with FreeRTOS.  It really has no dependency on the OS.  I also use heap4 so that shouldn't be an issues as long as you have sufficient heap. The LFS_THREADSAFE feature should only be needed if you are accessing the filesystem from multiple tasks.  In general with a multitasking OS such as FreeRTOS you will need that but in a simple testing scenario where you are just trying to get it working with a single task then it shouldn't matter.  In my test application for lfs and FreeRTOS I am not using LFS_THREADSAFE because it is only used from one task and there are no there tasks using the underlying resources such as the Flash and SPI. If you are accessing files from multiple tasks you will need that.  But be aware that other resources such as your underlying block device might be accessed from other tasks (even if lfs isn't) in which case you might need to serialize access using a mutex - otherwise another task might try to access the block device at the same time as lfs. I hadn't updated my lfs in over a year but it looks like it hasn't changed since then. I've just not had any problems with lfs.  Most problems I have with freertos is because my task stack is too small which leads to memory corruption or crashing. Matt

On Tuesday, March 8, 2022, 01:52:40 AM EST, deslim01 ***@***.***> wrote:  

Hi, I am having the similar problem too using FreeRTOS. Occasionally, I can mount little FS after trying to mount several times. However, most of the time I failed to mount it as what was being mentioned above. Even the formatting is not stable as well with more failure.

From the discussion above, I am using heap 4, and I don't think I use any LFS_THREADSAFE option at all. Should I enable the LFS_THREADSAFE options?

The lfs configuration is as follows which works in non-FreeRTOS environment.

const struct lfs_config LittleFS_config = { .context = (void*)&LittleFS_ctx, .read = lfs_mflash_read, .prog = lfs_mflash_prog, .erase = lfs_mflash_erase, .sync = lfs_mflash_sync, .read_size = 16, .prog_size = 256, .block_size = 4096, .block_count = 1024, .cache_size = 256, .block_cycles = 200, .lookahead_size = 16 };

And the underlying block device driver has been tested working fine.

I follow the examples where I try to mount before format, and if it failed, I will format then remount it. Yet same thing happened. And I tried to single step through it but when it hit the lfs_raw, some how, it just return zero,,

Hope you are able to help on this. Thanks.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.***>

[desmond-lim-tfs]

Hi Matt,

Thank you for your reply. In the case if there are multiple task sharing the same resources like SPI and Flash, does it mean I just need to enable the LFS_THREADSAFE option by defining the macro?

There are a few questions I have in mind for the lfs.

1. When you mention there is no problem for you, does it mean that you can mount your lfs 100% successfully or intermittently it fail to work and you need to format it?
2. I have partitioned the SPI or Flash specifically for this lfs to work in that region for testing purposes. I only partition 1MByte for the space because I was using another Flash sector for something else. From the configuration above with the block size of 4096 and block count of 256 (I have changed this from 1024 to 256), this should be about right.. Yet, I have experiencing corruption error when mounting and formatting. Is there something I have missed out in this case?
3. My idea of partitioning the Flash and allowing certain sector to be used for lfs I think should be ok right?
4. If it is ok, when I am doing mounting or formatting, will it format the other sector as well? I try to step through the code but it takes quite a fair bit of time to understand what the code is really doing in the rawmount function. Any insights on this?

That's all from me.

[desmond-lim-tfs]

Hi Matt,

Additional question suddenly pops up in my mind.

5. In the configuration, read_size = 16 means able to read 16 bits right? If I change it to 8, then, it will read a single byte, am I correct? That goes with prog_size / write size =256 bytes right? Any problem with my configuration above?

Cheers, Desmond

[maboytim]

Hi Desmond, If you SPI and Flash are shared below the LFS layer then using LFS_THREADSAFE win general will not be enough. For example, assume you are using an SPI Flash for the LFS BD.  If there are other devices attached to the same SPI then if an LFS file is being accessed from one task and the other SPI device is being accessed from another task then it's possible that both tasks could be using the SPI at the same time and they would interfere with each other and have very unpredictable results - in which case you would want to protect the SPI with a mutex. Likewise, if the SPI Flash used for the LFS BD is also used for things other than the LFS then if an LFS file is being accessed from one task and another task is making a direct access to the SPI Flash then it's possible that both tasks could be using the SPI Flash at the same time and they would interfere with each other and have very unpredictable results - in which case you would want to protect the Flash with a mutex. And finally, if you have multiple tasks accessing LFS files then you would want to protect the LFS with a mutex (using the LFS_THREADSAFE options). So in this scenario you could have three mutexes - one at the SPI driver level, one at the Flash driver level, and one at the LFS level. If the LFS is the only thing that uses the Flash and the Flash is the only thing connected to the SPI then using just LFS_THREADSAFE would be sufficient because the only way to get to the Flash and to the SPI in that case is through the LFS layer. Our SPI and Flash driver init functions have options to use a mutex or not.  The application developer knows if multiple devices and multiple tasks will be using the SPI and the Flash - if so then it will enable the mutex, otherwise it doesn't need to.  Our flow is typically open_flash->open_spi->close_spi->close_flash.  If the mutex is enabled then it will be obtained in the open and released in the close so we know that at most one task can be using them at a time. Regarding your questions 1) Yes the LFS filesystem is remounted successfully each time.  I have firmware images stored in files which must be downloaded to their corresponding devices on startup and that works everytime.2) It is okay to partition the Flash for the LFS an other things.  The partitions should be non-overlapping of course.  This will work fine - I often have a FatFS and LFS in the same SPI Flash.  However, just partitioning isn't sufficient if the partitions are accessed by different tasks, you need the mutex.  For example one task could be trying to read an LFS file while another erasing a block and the Flash can't do both at the same time.3) Yeah as I said tat's fine, and a common use case.4) To be honest I don't know the details of the inner workings of LFS - nor do I really want to.  Either it works or likely I am unable to fix it.  Sometimes it is useful to step into the code to if for example it is failing to see how or why it is failing.  This is typically though more to understand what I am doing wrong than to identify what it is doing wrong.  Also keep in mind that it is sometimes difficult to step through an application that uses multiple tasks and interrupts.  If you want to see what's going on I would add a lot of printf's.5) The configuration settings are pretty well described in the config struct definition.  I guess it's not clear if read_size is bits or bytes but I assume it's bytes.  Nothing jumped out at me as wrong with your configuration but I am by no means an expert on LFS - I just used it and it seems to work.  In contrast I had tried SPIFFS and that was a trainwreck - I don't know how anyone can use that. What processor and toolcahin are you using? Matt On Tuesday, March 8, 2022, 09:19:30 PM EST, deslim01 @.***> wrote:

Hi Matt,

Thank you for your reply. In the case if there are multiple task sharing the same resources like SPI and Flash, does it mean I just need to enable the LFS_THREADSAFE option by defining the macro?

There are a few questions I have in mind for the lfs.

• When you mention there is no problem for you, does it mean that you can mount your lfs 100% successfully or intermittently it fail to work and you need to format it?
• I have partitioned the SPI or Flash specifically for this lfs to work in that region for testing purposes. I only partition 1MByte for the space because I was using another Flash sector for something else. From the configuration above with the block size of 4096 and block count of 256 (I have changed this from 1024 to 256), this should be about right.. Yet, I have experiencing corruption error when mounting and formatting. Is there something I have missed out in this case?
• My idea of partitioning the Flash and allowing certain sector to be used for lfs I think should be ok right?
• If it is ok, when I am doing mounting or formatting, will it format the other sector as well? I try to step through the code but it takes quite a fair bit of time to understand what the code is really doing in the rawmount function. Any insights on this?

That's all from me.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.***>

[desmond-lim-tfs]

Hi Matt,

I am still exploring on this LittleFS thingy and with the minimum documentation and a lot of people facing similar corruption issue, what I can say is I will try to find what the heck is going on then. Anyway, I can understand your points and I really appreciate that you explain that in such thoroughness.

The processor I am using is NXP iMX and using the NXP MCU toolchain in MCUExpresso.

They have a SDK which work perfectly, however, after I ported it into my own projects, problems started to surface.

To be honest, my take on the mutex usage, I think there is no harm to enable that all the time because the no matter if the resources is shared or not, mutex will somehow protected it.. Oh, I was reading on the SPEC.md, it mentioned that for a single 4-byte files, additional 32k bytes are used. Is that the case for you?

Cheers,

[maboytim]

Hi Desmond, In my applications I usually use fatfs for convenient transfers with windows.  The files aren't mission critical so it's not catastrophic if the filesystem gets corrupted.  However my current application does rely on the files so I planned to use fatfs during development and later switch to lfs.  For testing I used the lfs fuse driver for linux to copy files to the lfs and then manipulated the files within my application and everything seemed okay.  I can certainly reboot and remount the lfs and load my firmware files without any problem or corruption. However I did more extensive write testing and it seems after some number of writes the filesystem does become corrupt.  Note that this is using the same block device driver as the fatfs which has worked flawlessly for a year on this project and for years on other project. So I don't think the issue that I am seeing is the same as your issue but I will have to investigate it further.  I wanted to mention this because I don't want to mislead you into thinking it's okay when maybe it's not.  It certainly could be user error at this point. Regarding the mutex it shouldn't hurt to have it even if you don't need it.  Just be aware though that the mutex at the lfs layer may not be sufficient to protect the block and SPI layers depending on how they are used. The lfs in the mbed source tree might be different than the separate mbed module - I seem to recall at one time they were out of sync.  Also I think I updated lfs since my original testing so it's possible that introduced a bug. In any case keep us informed of your findings and I will do the sae. Matt On Wednesday, March 9, 2022, 01:34:04 AM EST, deslim01 @.***> wrote:

Hi Matt,

I am still exploring on this LittleFS thingy and with the minimum documentation and a lot of people facing similar corruption issue, what I can say is I will try to find what the heck is going on then. Anyway, I can understand your points and I really appreciate that you explain that in such thoroughness.

The processor I am using is NXP iMX and using the NXP MCU toolchain in MCUExpresso.

They have a SDK which work perfectly, however, after I ported it into my own projects, problems started to surface.

To be honest, my take on the mutex usage, I think there is no harm to enable that all the time because the no matter if the resources is shared or not, mutex will somehow protected it.. Oh, I was reading on the SPEC.md, it mentioned that for a single 4-byte files, additional 32k bytes are used. Is that the case for you?

Cheers,

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.***>