How to import littleFS in my project without mbedOS

[masterLef]

Dear all,

I use the nRF52840 DK with Segger Studio and a Micron SPI SLC NAND 2Gb (256MB) Flash memory. I have already implemented the basic write read and erase functions for the NAND chip and they work fine. Now, I want to use the flash memory chip with files, wear leveling etc so, I decided to go with littleFS although I have not used a file system for my projects before.

The problem is that I do not use mbed OS and I want to use littleFS without the mbed drivers (i.e. for SPI comm etc) since I use the nRF52 SDK drivers to talk to the chip. After searching for the modifications I had to do in littleFS files by reading the code, a few fundamental (newbie) questions came up that I cannot answer.

1. In order to use littleFS in my project I must bind LittleFS with the NAND functions I have wrote for read/write, erase etc. Correct? If yes, what part should I modify in the littleFS code?
2. I noticed that in the given example of readme.md there are four block device operations that must be provided by the user. So, if I understand right, this is the point where I import the read,write and erase functions that I have already wrote for the NAND flash memory. Correct?
3. I found the function pointers for these block device operations in lfs.h file, inside lfs_config structure but I do not understand what the lfs_off_t off argument is used for and how I should use it. Also, the implementation of these functions' arguments do not match with the functions' arguments that I have created so I suppose that I have to re-write them to match exactly with the function pointers or should I change something else in lfs.h file
4. Is there something else that I need to change/add in littleFS files so that I can use it with my controller and NAND memory?

Thank you for your time

Best Regards

[geky]

Hi @masterLef, glad for the interest.

As you've noted the main thing that's harder outside Mbed OS (or conversely, what Mbed OS makes easier) is the block device drivers for various hardware.

The key thing will be setting up the config struct passed to littlefs to correctly match your hardware: https://github.com/ARMmbed/littlefs/blob/ce2c01f098f4d2b9479de5a796c3bb531f1fe14c/lfs.h#L151-L200

Along with the bd functions, you also need to configure the geometry (read_size, prog_size, erase_size) to match your hardware. These are best when set to the smallest size the bd supports. You should be able to find these sizes in your datasheet, though they may use different names.

1. In order to use littleFS in my project I must bind LittleFS with the NAND functions I have wrote for read/write, erase etc. Correct? If yes, what part should I modify in the littleFS code?

Yep. You shouldn't actually need to modify littlefs since these functions are passed to littlefs as a part of the config structure.

2. I noticed that in the given example of readme.md there are four block device operations that must be provided by the user. So, if I understand right, this is the point where I import the read,write and erase functions that I have already wrote for the NAND flash memory. Correct?

Yep. Note the sync function is only needed if your NAND device has a cache that needs to be flushed to insure the data is stored in persistent storage. Otherwise sync can be a noop and return 0.

3. I found the function pointers for these block device operations in lfs.h file, inside lfs_config structure but I do not understand what the lfs_off_t off argument is used for and how I should use it. Also, the implementation of these functions' arguments do not match with the functions' arguments that I have created so I suppose that I have to re-write them to match exactly with the function pointers or should I change something else in lfs.h file

You may need a wrapper to convert the arguments from littlefs to the arguments your bd driver expects.

The reason for the off argument is because littlefs refers to blocks by block addresses instead of by byte addresses. The block argument indicates the current block and the off argument specifies the byte address inside the block.

So if you need the byte-level address you can do this:

byte_off = block*block_size + off

Note that this could lead to 32-bit overflow, but you most likely don't care unless you are dealing with gigabytes of storage.

4. Is there something else that I need to change/add in littleFS files so that I can use it with my controller and NAND memory?

I think that's it, hopefully the above is useful.

It's been requested multiple times to add a PORTING.md or similar, and I think it's important but I haven't been able to work on this yet. Seeing the questions a new user raises is useful feedback for this though so thanks for raising and issue!

[masterLef]

Hi Geky,

Thank you for the detailed reply. It is really useful!

The NAND Flash memory chip is programmed and read in page-based operations and It is erased in block-based operations.

According to datasheet: a. 1 Partial Page = 1/4 of a Page = 512B b. 1 Page = 2048 bytes + 128 bytes (where 128 bytes are used as spare area) c. 1 Block = 64 pages d. 1 Plane = 1024 Blocks e. Whole Flash = 2 Planes = 2048 Blocks

So if I get this right( please correct me if not), the geometry in the lfs_confic should be

• read_size = 2048 B ( since the chip is read in page-based operations (b) )
• prog_size = 2048 B ( since the chip is programmed in page-based operations (b) )
• block_size = 2048 B/page 64 pages/block ( bc)
• block_count = 2048 B (e)
• block_cycles = according to the datasheet block endurance is <10000. So I suppose that I can set it to 1000 (or more?)

Again, Thank you for your time!

[geky]

That looks right. I wonder what the partial-page is useful for?

We don't have a good metric for choosing block_cycles yet. It decides when blocks need to be relocated as a part of wear-leveling, so it may work better in the 100-500 cycle range for a block device with ~10K erase cycles per block.

Note also there are some performance issues with littlefs, which can become problematic on NAND because of how large the geometry of the storage is.

[masterLef]

I wonder what the partial-page is useful for?

I haven't used it but according to the NAND flash chip datasheet, "Due to the large size of NAND Flash pages, partial-page programming is useful for storing smaller amounts of data. Each NAND page can accommodate four PC-sized, 512-byte sectors.

The spare area of each page provides additional storage for ECC and other software information.While it is advantageous to write all four sectors at once, often this is not possible. For example, when data is appended to a file, the file might start out as 512 bytes, then grow to 1,024 bytes. In this situation, a second PROGRAM PAGE operation is required to write the second 512 bytes to the NAND Flash device.

The maximum number of times a partial page can be programmed before an ERASE is required is eight; this accommodates four sectors of data and four sectors of ECC, each programmed separately."

[geky]

Oh interesting. It's only an optimization but you could actually use the smaller 512-byte size as the read/prog size.

It littlefs has >prog size bytes to write, it can call the bd functions with a multiple of the prog size. So you can still take advantage of the larger 2KiB prog size if, say, the operation is faster.

[masterLef]

Ok so littleFS will handle a possible larger load (>512) by its own by setting a multiple value, if I get it right?

[geky]

Yes, it may program 512 bytes, 1024 bytes, or 1536 bytes, but nothing that's not divisible by 512. I should have mentioned that earlier.

[masterLef]

Ok good! So it makes sense to set it to 512 bytes. I will come back as soon as I make it work.

Thank you very much for your help Geky

[fengchun1]

Hi @masterLef, glad for the interest.

As you've noted the main thing that's harder outside Mbed OS (or conversely, what Mbed OS makes easier) is the block device drivers for various hardware.

The key thing will be setting up the config struct passed to littlefs to correctly match your hardware: https://github.com/ARMmbed/littlefs/blob/ce2c01f098f4d2b9479de5a796c3bb531f1fe14c/lfs.h#L151-L200

Along with the bd functions, you also need to configure the geometry (read_size, prog_size, erase_size) to match your hardware. These are best when set to the smallest size the bd supports. You should be able to find these sizes in your datasheet, though they may use different names.

1. In order to use littleFS in my project I must bind LittleFS with the NAND functions I have wrote for read/write, erase etc. Correct? If yes, what part should I modify in the littleFS code?

Yep. You shouldn't actually need to modify littlefs since these functions are passed to littlefs as a part of the config structure.

1. I noticed that in the given example of readme.md there are four block device operations that must be provided by the user. So, if I understand right, this is the point where I import the read,write and erase functions that I have already wrote for the NAND flash memory. Correct?

Yep. Note the sync function is only needed if your NAND device has a cache that needs to be flushed to insure the data is stored in persistent storage. Otherwise sync can be a noop and return 0.

1. I found the function pointers for these block device operations in lfs.h file, inside lfs_config structure but I do not understand what the lfs_off_t off argument is used for and how I should use it. Also, the implementation of these functions' arguments do not match with the functions' arguments that I have created so I suppose that I have to re-write them to match exactly with the function pointers or should I change something else in lfs.h file

You may need a wrapper to convert the arguments from littlefs to the arguments your bd driver expects.

The reason for the off argument is because littlefs refers to blocks by block addresses instead of by byte addresses. The block argument indicates the current block and the off argument specifies the byte address inside the block.

So if you need the byte-level address you can do this:

byte_off = block*block_size + off

Note that this could lead to 32-bit overflow, but you most likely don't care unless you are dealing with gigabytes of storage.

1. Is there something else that I need to change/add in littleFS files so that I can use it with my controller and NAND memory?

I think that's it, hopefully the above is useful.

It's been requested multiple times to add a PORTING.md or similar, and I think it's important but I haven't been able to work on this yet. Seeing the questions a new user raises is useful feedback for this though so thanks for raising and issue!

Hi Geky, I have a question (What is the flash address corresponding to the block address）

[masterLef]

Hi @fengchun1,

Be more specific please because the way you say it, does not make sense to me.

There is no relation between Flash address and Block address because flash address is the address the micro-controller will send through the SPI/I2C communication to the flash chip before any other data transmission with it ( to indicate which chip it wants to "talk" to). Block Address begins from 0 up to whatever the size of your flash chip can be. Flash Datasheet has all the info you may need.

[fengchun1]

Hi Geky, Thank you for reply. For example: I have a flash, I want to allocate some memory for lfs, how do I determine where to allocate it? Below is my code: int _block_read(const struct lfs_config c, lfs_block_t block, lfs_off_t off, void buffer, lfs_size_t size) { STMFLASH_Read ( block c->block_size + off, (uint8_t )buffer, size );
//flash_read_block(block c->block_size + off, (kal_uint8 )buffer, (kal_uint32)size); return 0; } block * c->block_size + off 。Which address does this value point to? Thank you very much for your help Geky

[e107steved]

how do I determine where to allocate it? You don't 'determine' - you have to decide. Block 0 offset 0 address is whatever you decide is the lowest address in your flash block; then count up from there.

[fengchun1]

Hi @e107steved , Thank you for reply. I use the internal flash, can I add a base address on this basis? addr = block * c->block_size + off+BASEaddr

[thrasher8390]

@fengchun1, my understanding is that as long as your BASEaddr is the start address of a sector/block (smallest portion of flash that can be erased) then it will work. LFS doesn't care where the memory actually ends up. All it cares about is that it can read, write, and erase a sector/block. I've added this partitioning idea to my project and it's been working as expected.

[ShaharHD]

@masterLef If you're talking about Nordic SDK with littlefs, I have the littlefs running on the PCA10056 (nRF52840 DevKit) and a custom nRF52840 board (with a different QSPI Flash) with no issues.

You'll want to use the nrfx_qspi library, and implement the little functions and map them to the QSPI functions

Few notes:

1. nrfx_qspi_init make sure to avoid using the handler as you want blocking mode
2. The QSPI functions require address, which is easy to calculate in read/write: uint32_t addr = c->block_size * block + off; and in the erase: uint32_t addr = c->block_size * block;
3. The sync function (as everything is done in blocking mode) can be simply return LFS_ERR_OK;

As for you flash, you'll might want to look at the following: look for nrf_serial_flash_params.h and it's counter part nrf_serial_flash_params.c You can use that example and the nrf_block_dev_qspi.c code as an example on how to identify your flash.

[masterLef]

@geky @ShaharHD @thrasher8390 Hi guys, after a long time I resumed the implementation of write and read functions for the config struct passed to littlefs for bonding it with my embedded system. I am using Micron MT29F 2Gbit and 8 Gbit NAND Flash chips. Currently I implement the write function (wrapper function with bare-metal SPI communication function inside it for the Flash chip) and I have a few questions because I am a little stuck regarding the low-level corner cases where littleFS cannot intervene. First of all if I get it right the LFS wrapper function for writing data to Flash needs the following parameters :

• block_Number
• Offset
• Buffer that holds data to be written
• Amount of data to be written (size)

Question: Do I need to implement block change inside the write wrapper function in case the size of the data to be written are near a block limit?

For example, if LittleFS will start writing in the last page of a block (i.e. page 63 of block 47) and the size of data to be written are 3kBytes (and flash page size is 2048 Bytes), the first 2048kB of data will be written in page 63 of block 47 and then the block number must be increased by one and the page number to drop to zero so that the remaining 1024 Bytes to be written to the page 0 of block 48). Do I need to implement the logic for block change or LittleFS does that for me?

[geky]

Question: Do I need to implement block change inside the write wrapper function in case the size of the data to be written are near a block limit?

Nope, writes to different blocks will be different cfg->erase/cfg->prog calls. Blocks are the building-blocks of LittleFS, so it only ever writes at most one block at a time.

If the data exceeds a single block, LittleFS will split it into two blocks, which may be nowhere near each other.

[masterLef]

Thank you for the support Geky.

[marcosgabrielserrano]

@masterLef

I am about to work on the same device(Micron MT29F 2Gb) with write over spi.

Do you have sample code of your implementation or is it closed source?

Also, what did you set for the lookahead_size and cache sizes?(Assuming it hasn't changed since above post)