littlefs-project / littlefs

A little fail-safe filesystem designed for microcontrollers
BSD 3-Clause "New" or "Revised" License
5.13k stars 791 forks source link

[Question]: Fragmentation - Are files always allocated in a compact block sequence? #878

Open Jierr opened 1 year ago

Jierr commented 1 year ago

Heya, I read through the littlefs/DESIGN.md but could not answer all of my questions.

1) Are larger files (>1/4 block size) always stored as a compact sequence of blocks (One big blob) or can files be stored partially in multiple block sequences like: fileA_part1 -> fileB -> fileA_part2.

The CTZ skip-lists may already hold the answers (not obvious to me)

I am asking because I want to grasp the amount of fragmentation I could expect on a relatively small EEPROM storage over time, when adding/writing/deleting files.

2) In this context a follow up question arises. If a file grows in size and breaches its allocated blocks and there are no free block sequences available to hold the file as one blob, will the file be split automatically?

geky commented 1 year ago

Hi @Jierr, good question.

littlefs is a strictly block-oriented filesystem, that is, littlefs only manages blocks and files larger than a block are always broken up into multiple blocks. This decision simplifies the filesystem's design and avoids most fragmentation issues.

The CTZ skip-lists may already hold the answers (not obvious to me)

From a high-level you can think of the CTZ skip-list as a more complicated/efficient linked-list, where each node is a block. If you write files out of order, littlefs is perfectly happy to interleave them, because at this level it's not really thinking of these as contiguous files any more:

             fileB       fileA
               |           |
               v           v
.-----.-----.-----.-----.-----.-----.-----.
|fileA|fileB|fileB|fileA|fileA|     |     |
|part1|part1|part2|part2|part3|-->  |     |
|     |     |     |     |     |     |     |
'-----'-----'-|---'-|---'-|-|-'-----'-----'
   ^     ^----'     | ^---' |
   '----------------'-------'

In this context a follow up question arises. If a file grows in size and breaches its allocated blocks and there are no free block sequences available to hold the file as one blob, will the file be split automatically?

In a sense, yes, but it may be more correct to think of the file already being split, even if the contents happen to be contiguous.

Jierr commented 1 year ago

Thanks for the clarification! I think this info makes the system more versatile, than I initially thought.