implementation of a wear leveling file system for NOR serial flash- is Littlefs better way? way to implement littlefs!!!

[KRekha3]

Hi How to implement a wear leveling file system for NOR serial flash? is Littlefs better way? if yes how to use littlefs to create a filesystem?

[lsilvaalmeida]

Hello @KRekha3

LittleFS is at least a good way to do it. In the README you will find an example of implementation using an emulated block device. To adapt it to your scenario you should create your own wrap functions for read, prog, erase and sync and change it in the configuration struct.

Once you have mount littlefs, it will provide you a set of POSIX-like commands that you can use in your API.

If you want more help, let us know more about your problem (your description wasn't so clear).

[KRekha3]

Thanks for the information. I wanna implement flash filesystem for STM32L151X board through SPI lines. What are the set up files that I need to include in my project , what tools should be used to compile and which OS? Can you please suggest...

[lsilvaalmeida]

You have to write, at least, the driver functions to prog, read, etc. You can find how in manufactor's site.

what tools should be used to compile and which OS

You can use no matter what to compile, since this supports c99, and as OS. But littlefs was developped to works with MbedOS, so if you are free to chose it, I recommend that you take this one. You can find all the projects files here https://github.com/armmbed/mbed-os.

[KRekha3]

Hi Do we have any existing example code for Littlefs implemented on STM32? need your support ASAP...

[geky]

Related https://github.com/ARMmbed/littlefs/issues/196

[KRekha3]

Do you have any extisting wear leveling filesystem implemented on STM32?

On Tue, May 21, 2019, 6:52 PM lsilvaalmeida notifications@github.com wrote:

You have to write, at least, the driver functions to prog, read, etc. You can find how in manufactor's site.

what tools should be used to compile and which OS

You can use no matter what to compile, since this supports c99, and as OS. But littlefs was developped to works with MbedOS, so if you are free to chose it, I recommend that you take this one. You can find all the projects files here https://github.com/armmbed/mbed-os http://url.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ARMmbed/littlefs/issues/183?email_source=notifications&email_token=AMDC2CE56KNKTK5MJJBRJITPWPZRLA5CNFSM4HNUDJGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV34K7Q#issuecomment-494388606, or mute the thread https://github.com/notifications/unsubscribe-auth/AMDC2CHTJEDKI3QWNF7ZLEDPWPZRLANCNFSM4HNUDJGA .

[geky]

littlefs should work. I've used it on an STM32 with Mbed OS, but I don't know of an example out side of Mbed OS. https://github.com/pellepl/spiflash_driver looks the most promising but I have not used it.

[KRekha3]

I recommend that you take this one. You can find all the projects files here https://github.com/armmbed/mbed-os.

Hi there are many files. what files need to use to create a project file for littlefs implementation.??? can you please help me. because i found many folders inside Littlefs folder which has too many option to test it. how to test the littlefs application?? do you have any test steps or any project(IAR toolchain) created specifically for littlefs implemetation? And where are you mounting the filesystem i mean is it a flash memory ?

[KRekha3]

Hi @lsilvaalmeida, i am trying to implement littlefs and i have added files manually to IAR to create a project. i am getting compiler error attached image. I don't have any idea in which file those are defined.

i have taken main.c from : mbed-os-master\features\storage\filesystem\littlefs\TESTS\filesystem\files from your link

please suggest your solution

[geky]

Hi @KRekha3, they filesystem example may be a good place to start: https://github.com/armmbed/mbed-os-example-filesystem

Though note it does bring in a full OS.

[KRekha3]

Hi @geky your link was really helpful. I have got more insight of filesystem implementation now.. Thank you .I will follow the link to achieve filesystem implementation

[KRekha3]

Hi @geky your link was really helpful. I have got more insight of filesystem implementation now.. Thank you .I will follow the link to achieve filesystem implementation

[KRekha3]

Hi I have tried to implement Littlefs. but unable to understand block configuration parameters and i am getting an error block != 0x0ffffffff. can you please explain why this error occurs? Block configurations are done w.r.t MX25R6435F flash memory.

attached code snippet for your reference.

Thanks ![Uploading block error.PNG…]()

``

[lsilvaalmeida]

I don't know if there is the source of this problem, but your configuration don't work: the lookahead buffer shall be lookahead sized and 64-alligned.

[geky]

@lsilvaalmeida is right, the lookahead buffer needs to match the lookahead_size parameter:

- uint8_t file_lookahead_buf[16];
+ uint8_t file_lookahead_buf[256];

This changed recently, is documentation incorrect somewhere?

[KRekha3]

ok , thanks for pointing it out @lsilvaalmeida . I have made the modifications to the buffer. However, still I am facing the same error. 64 bit aligning is not something which I have done earlier and trying to do it now. Below is the snippet of code currently being used to test the filesystem using SPI device driver without a fancy block device driver.

`int user_provided_block_device_read(const struct lfs_config c, lfs_block_t block, lfs_off_t off, void buffer, lfs_size_t size) { printf("read\n"); MEM_Read((block c->block_size + off), (uint8_t)buffer, size); return 0; }

int user_provided_block_device_prog(const struct lfs_config c, lfs_block_t block, lfs_off_t off, const void buffer, lfs_size_t size) { printf("prog\n"); MEM_Write((block c->block_size + off), (uint8_t)buffer, size); return 0; }

int user_provided_block_device_erase(const struct lfs_config c, lfs_block_t block) { printf("erase\n"); MEM_SectorErase(block c->block_size); return 0; }

int user_provided_block_device_sync(const struct lfs_config *c) { printf("sync\n"); return 0; } uint8_t lfs_read_buf[256]; uint8_t lfs_prog_buf[256]; uint8_t lfs_lookahead_buf[256]; // 128/8=16 uint8_t lfs_file_buf[256];

// configuration of the filesystem is provided by this struct const struct lfs_config cfg = { // block device operations .read = user_provided_block_device_read, .prog = user_provided_block_device_prog, .erase = user_provided_block_device_erase, .sync = user_provided_block_device_sync,

// block device configuration
.read_size = 256,
.prog_size = 256,
.block_size = 32768,
.block_count = 256,
.cache_size = 256,
.lookahead_size = 256,

.read_buffer = lfs_read_buf,
.prog_buffer = lfs_prog_buf,
.lookahead_buffer = lfs_lookahead_buf

}; `

[KRekha3]

Hi, I have modified uint8_t lfs_lookahead_buf[] to make it 64bit aligned.

unable to mount filesystem. i am getting error in below line(LFS_ERR_CORRUPT)

[lsilvaalmeida]

In the config struct, you should give the function address for the callback. I mean,

...
.read = &user_provided_block_device_read,
...

I think that lfs can't read the device so it see as corrupted.

For the lookahead buffer, just change the attribute to 64 like __attribute__(__aligned__(64))

[KRekha3]

Hi @lsilvaalmeida , as per your suggestion i have modified the code for passing address of the function in config struct and modified lookahead buffer alignment to 64. But still i am facing the same issues. what might be wrong?

[lsilvaalmeida]

It's normal have this problem in the first boot. So you have to format the fs and try again. Are you already doing this?

Look at the example below:

```
int err = lfs_mount(&lfs, &cfg);

// reformat if we can't mount the filesystem
// this should only happen on the first boot
if (err) {
    lfs_format(&lfs, &cfg);
    lfs_mount(&lfs, &cfg);
}

[KRekha3]

Yes i am already doing that. even after format , lfs_mount returns -84 error code which is nothing but LFS_ERR_CORRUPT.

[lsilvaalmeida]

lfs_format returns without any error?

[KRekha3]

Hi @lsilvaalmeida , lfs_format() is returning -28 as error code. lfs_dir_commit() is returning error code -28 which jumps out of lfs_format with return value -28 (LFS_ERR_NOSPC).

[lsilvaalmeida]

Try to debug it. I don't think so that is lfs_dir_commit who returns this error but lfs_init. Maybe your microcontroller don't support dynamic allocation and lfs do it by default.

[KRekha3]

@lsilvaalmeida sure i will try to debug and will come back on this. Thank you

[kewld]

Try to debug it. I don't think so that is lfs_dir_commit who returns this error but lfs_init. Maybe your microcontroller don't support dynamic allocation and lfs do it by default.

so if dynamic memory is not possible, @lsilvaalmeida can there be a static buffers provided to run littleFS?

[lsilvaalmeida]

Hello @kewld. Yes, you can pass static buffers to lfs. In the config struct, there is 3 optionals buffers (read, prog and lookahead) which you can allocate staticaly before the initialisation of this variable. The read and prog buffers shall match with cache size and the third one with lookahead size.

Also, for the file buffer, you can allocate staticaly a buffer and open a file with opencfg.

Here my code for this:

/* the file system's caches are always zeroed in the mount so
 * we dont have to store it in the uninit*/
UINT8 read_buffer[LFS_CACHE_SIZE];
UINT8 prog_buffer[LFS_CACHE_SIZE];

/* the lookahead vector have to be 64-aligned and we ensure it
 * with this attribute macro */
UINT8 __attribute__((aligned(64))) lookahead_buffer[LFS_LOOKAHEAD_SIZE];

...

/* configuration struct for the FS. More info on the lfs header */
const struct lfs_config cfg = {

    ...

    // static buffers configuration
    .read_buffer = read_buffer,
    .prog_buffer = prog_buffer,
    .lookahead_buffer = lookahead_buffer
};

And for open a file with a static buffer:

        /* file cache */
    const struct lfs_file_config file_cfg = {
        .buffer = &file_buffer
    };

    /* open the file in read/write mode or create it if doesn't exists */
    int l_i32_error = lfs_file_opencfg(p_lfs, p_file, path, LFS_O_RDONLY | LFS_O_CREAT, &file_cfg);

I hope that this can help you.

[KRekha3]

Hi @lsilvaalmeida @geky , i have modified code according to your suggestion but still the lfs_format() fails. below is the snippet where the error is returned : in lfs.c - lfs_dir_commitcrc()

` // write out crc uint32_t footer[2]; footer[0] = lfs_tobe32(tag ^ commit->ptag); commit->crc = lfs_crc(commit->crc, &footer[0], sizeof(footer[0])); footer[1] = lfs_tole32(commit->crc); err = lfs_bd_prog(lfs, &lfs->pcache, &lfs->rcache, false, commit->block, commit->off, &footer, sizeof(footer)); if (err) { return err; } commit->off += sizeof(tag)+lfs_tag_size(tag); commit->ptag = tag ^ (reset << 31);

// flush buffers
err = lfs_bd_sync(lfs, &lfs->pcache, &lfs->rcache, false);
if (err) {
    return err;
}

// successful commit, check checksum to make sure
uint32_t crc = 0xffffffff;
lfs_size_t size = commit->off - lfs_tag_size(tag) - commit->begin;
for (lfs_off_t i = 0; i < size; i++) {
    // leave it up to caching to make this efficient
    uint8_t dat;
    err = lfs_bd_read(lfs,
            NULL, &lfs->rcache, size-i,
            commit->block, commit->begin+i, &dat, 1);
    if (err) {
        return err;
    }

    crc = lfs_crc(crc, &dat, 1);
}

if (err) {
    return err;
}

if (crc != commit->crc) {
    return LFS_ERR_CORRUPT; //Error returned
}`

[FreddieChopin]

For the lookahead buffer, just change the attribute to 64 like __attribute__(__aligned__(64))

The value for alignment is 64 bits, which is 8 bytes. The argument for aligned() attribute is in bytes, therefore it should be just 8, not 64.

[KRekha3]

HI @FreddieChopin thanks for the clarification. previously my code was aligned with value 8, later on based on comments i changed it to 64. Now i have made it 8 as per clarification. but still there is no luck in solution. i am still stuck with the same issue as explained in my previous comment. need help in resolving this issue.

Thanks Rekha

[Johnxjj]

HI @lsilvaalmeida, I remember writing in the comment to the lfs_file_opencfg function "The configuration must be assigned when the file is opened." I think the configuration structure here refers to lfs_file_config instead of lfs_config. So it may not be necessary to use the lfs_file_opencfg function. Lfs_file_open should also work.

[KRekha3]

@Johnxjj , You are correct i had removed lfs_file_opencfg() and used only lfs_file_open() and it works.

[lsilvaalmeida]

Hi @Johnxjj. Yes, you're right. I was just saying in the case where you want to pass a static buffer for the file cache, you have to use open config (to not use malloc for the allocation), but it's not necessary.

[Johnxjj]

@lsilvaalmeida Yeah, you are right. If we don't want to use malloc for the allocation, we must define a static buffer for the file cache. I also find The lfs_file_open() calls lfs_file_opencfg() and assigns a struct lfs_file_config defaults by default, but does not define a buff, so malloc is required.

[KRekha3]

Hi @geky, @johnxjj, @lsilvaalmeida , i am trying to add attributes to lfs. i am able to succeed with lfs_setattr(), but unable to get written attribute values using lfs_getattr(). please see my open issue - #208 for more details.

[KRekha3]

i have resolved issue with attributes..

[KRekha3]

@geky , i have a small change in lfs usage: i want to mount and read file , i do not need other functions like write close and unmount features. so if i remove write, lfs_file_flush from lfs.c will this affect my code. can i mount a filesystem without using lfs_dir_compact() , lfs_dir_commit()?

[geky]

Hi @KRekha3, glad you were able to figure out the attribute issue, sorry for late responses. Was the issue because attributes aren't written out until you call lfs_file_close?

Your second question sounds related to https://github.com/ARMmbed/littlefs/issues/162

If you're interested in modifying littlefs, I think you should be fine removing lfs_file_flush. That is the one piece that brings in the writing logic of littlefs. I haven't tested this myself, but if you get it working let us know.

[KRekha3]

Thanks for your response. attribute issue was with file_close and also i was not printing value properly.

[KRekha3]

sure i will try removing lfs_file_flush() and will let you know the results.

[KRekha3]

Hi @geky , I have modified LFS to read only purpose. right now LFS code size is reduced almost to 5 Kbytes and we have tested for the functionality and it works fine without any barriers. Thank you for your constant support.

[KRekha3]

i have an issue w.r.t attributes. i have written a file to flash using LFS, then i opened file then did lfs_setattr() for the same file and closed file. now when reboot and read the attribute value its a garbage value which is unexpected. what is the error here? as i see set attribute writes value to flash memory in space. then why is it retrieving garbage value instead of written value???

[KRekha3]

i have an issue w.r.t attributes. i have written a file to flash using LFS, then i opened file then did lfs_setattr() for the same file and closed file. now when reboot and read the attribute value its a garbage value which is unexpected. what is the error here? as i see the lfs code for set attribute writes value to flash memory. is my understanding correct???

[geky]

i have written a file to flash using LFS, then i opened file then did lfs_setattr() for the same file and closed file. [...] as i see the lfs code for set attribute writes value to flash memory. is my understanding correct???

Ah, this sounds like it is because lfs_setattr does not update the attributes of open files.

lfs_setattr updates the disk, but if you have a O_WRONLY file open with the same attribute, it is left at its previous value (garbage). When you close/sync the file, it will update the attribute to the value in the struct lfs_attr (to garbage).

               on device          on disk       

open file ---> attr = garbage                   
                    |                         
lfs_setattr --------|-----------> attr = 1      
                    v                         
close file --> attr = garbage --> attr = garbage
                    |                  |      
                    v                  v      
result:        attr = garbage     attr = garbage

Maybe this should be changed so lfs_setattr updates all open files as well?

               on device          on disk 

                                  no attr 

open file ---> attr = garbage             

lfs_setattr -> attr = 1 --------> attr = 1
                    v                     
close file --> attr = 1 --------> attr = 1
                    |                  |  
                    v                  v  
result:        attr = 1           attr = 1

This raises a few questions

1. What does lfs_removeattr do to an open files with the attribute?
2. What do we do to attributes if a file is opened twice? If you close one file it does NOT update the other file's data. But should it update the other file's attributes? Or should be handle lfs_setattr different from open->close with struct lfs_attr?

[techNiv1]

@Johnxjj , You are correct i had removed lfs_file_opencfg() and used only lfs_file_open() and it works.

Hello @KRekha mam, can you please share a piece of code of how u solved and getting it work?

[KRekha3]

@Johnxjj , You are correct i had removed lfs_file_opencfg() and used only lfs_file_open() and it works.

Hello @krekha mam, can you please share a piece of code of how u solved and getting it work?

Hey @techNiv1, sorry I don't have that code base with me now. You can use any of the example code from git.