lfs_file_write following lfs_file_sync is really heavy operation

[petri-lipponen-suunto]

I have a code that saves data from streaming sensors for later retrieval. The backend is a 1 Gb NAND flash with blocksize of 128 kB (MT29). I've been experimenting with the regular flushing of the file to make sure that it is up-to-date in case something goes wrong and have bumped into following issue:

When the device has saved a largish amount of data to the flash, the lfs_file_write following lfs_file_sync takes a really long time (seconds). I've attached a log demonstrating the issue that shows the sync when about 40 kB has been written into the file. The sync itself is quite fast, but the lfs_file_write after the sync (lines 15-984) seems to consist of reading of 80 kB and writing 40 kB which obviously is a very heavy operation in 256 byte pieces over SPI.

I'm quite confused about the behaviour and would like to know more: is this my mis-understanding of the littlefs operation or is there a bug? I would not think that the FS needs to copy the whole file when flushing it...

I'm currently using LittleFS v.2.4.0 (from git tag) but noticed this already a version or two ago.

write_and_sync.log

[petri-lipponen-suunto]

I've studied this a bit further and found out that the _lfs_filewrite following _lfs_filesync seems to copy the last partial block of the file to a new block. Since block size is 128kB it will be really heavy operation. I've attached a log of a simple test case that I did. The test does:

1. Write to a file 10 bytes at the time and time the operation (TIME_DEBUGGER rows on the log).
2. Every 500 bytes call _lfs_filesync

As can be seen in the log the write following _lfs_filesync is small when file size is small or a bit over 131072 bytes, and gets progressively heavier on every sync towads the 128kB limit. At worst the write takes more than 2500 ms!

I'm contemplating calling the sync only when the file size is a bit over multiple of 128 kB (-8 bytes overhead?) but it sounds like a bit of a kludge. If there is any way to improve this in LittleFS, that would be great!

flush_test.txt

[petri-lipponen-suunto]

I also tried setting the metadata_max configuration (created by the https://github.com/littlefs-project/littlefs/pull/502) to 4 kB, but it did not affect the "write after sync" -performance.

[petri-lipponen-suunto]

Just found this comment in another issue:

https://github.com/littlefs-project/littlefs/issues/344#issuecomment-567195031

which explains this issue. I'm +1 for either of the suggested improvements since the current situation is quite difficult with large block size (= NAND chips).

For now I've implemented the lfs_file_sync call when going over to new block but I think that may cause wasting blocks (I haven't confirmed if that is the case).

[simon88]

Hi @petri-lipponen-suunto I have the same issue. When you say you sync every new block you mean you call sync_file every 128kB?

[geky]

@petri-lipponen-suunto, sorry for not responding, this flew under my radar.

The proposal in https://github.com/littlefs-project/littlefs/issues/344#issuecomment-567195031 is in the works, but is unfortunately a part of a larger set of changes which will take some time to land.

I have the same issue. When you say you sync every new block you mean you call sync_file every 128kB?

I'm curious if they figured it out, because the correct alignment is unfortunately excruciatingly complicated.

Calling sync after the first 128KiB works, but the next block will contain a pointer and have only 128KiB-4B of data. The math gets real ugly and is explained a bit more in DESIGN.md.

But let me emphasize, the math gets real ugly.

To avoid any wasted blocks, you need to sync when $\mathit{off}$ is zero, where:

$$ n = \left\lfloor \frac{N - \frac{w}{8} \left(\mathrm{popcount}\left(\frac{N}{B-2\frac{w}{8}}-1\right)+2\right)}{B-2\frac{w}{8}} \right\rfloor $$

$$ \mathit{off} = N - \left(B - 2\frac{w}{8}\right)n - \frac{w}{8}\mathrm{popcount}(n) $$

Where $N$ is the current file size, $B$ is the block size, and $w$ is the word size (32 bits).

For @petri-lipponen-suunto, this may have gone unnoticed because this ends up pretty close to just 128KiB. Overflow would result in copying only a couple of extra words in each block, but it does end up with more block erases than are necessary.

[simon88]

@geky nice, I'll try to syncas soon as I got zero offset, keep you in touch 👍