littlefs-project / littlefs

A little fail-safe filesystem designed for microcontrollers
BSD 3-Clause "New" or "Revised" License
5.15k stars 793 forks source link

How is superblock stored in LittleFS #651

Open madpeach96 opened 2 years ago

madpeach96 commented 2 years ago

The name is a bit of a misnomer. While the superblock entry serves the same purpose as a superblock found in other filesystems, in littlefs the superblock does not get a dedicated block. Instead, the superblock entry is duplicated across a linked-list of metadata pairs rooted on the blocks 0 and 1. The last metadata pair doubles as the root directory of the filesystem.

As per my understanding the superblock consists only of a name tag (magic string ("littlefs")) and inline-struct tag. To me both name tag and inline-struct tag looks permanent and doesn't need to be updated, which means there should be only one entry of superblock probably in blocks 0 and 1, so what does it mean to say the superblock entry is duplicated across a linked-list of metadata pairs.?

Also why do we need multiple superblock pairs?

geky commented 2 years ago

This is a great question! The short answer is to avoid the specific problem that a naive, traditional copy-on-write filesystem will very quickly kill whichever block the filesystem is rooted at.


If you look at a traditional copy-on-write update: image

(image borrowed from https://pthree.org/2012/12/14/zfs-administration-part-ix-copy-on-write)

All copy-on-write updates require a write to the root of the tree, in our case the superblock.

This sort of defeats the purpose of wear-leveling. Sure our leaf nodes are moved around in the storage, but writing to the root every write will quickly kill that specific block, rendering the filesystem unmountable.

littlefs's metadata is slightly different from traditional CoW, since we can write to metadata blocks multiple times (I've called this copy-on-bounded-writes (CoBW), maybe that will catch on). But this just delays the problem by a constant factor. Writes to anywhere in the filesystem still wear out the root very quickly.

The solution, at least in littlefs, is to treat the root like an anchor. If we write to the root too much, say block_cycles times, we increase the chain to our anchor by one by inserting a redundant superblock into a linked-list of superblocks leading to the actual root directory. This means we need to write to the filesystem block_cycles times before we have to update the original superblock again. And it will take block_cycles^2 write before we need to extend the anchor with another superblock.

It turns out the number of writes needed to update the original superblock grows extremely quickly (O(block_cycles^x), aka exponentially). So after a couple of extra superblocks, we never really need to update the original superblock. Avoiding an early death due to killing the anchor of the filesystem.


When we create these extra chain links, we create a full copy the original superblock. We don't have to, but as you noted this data is permanent, so there's no reason not to.

Some filesystems keep extra redundant superblocks around for recovery purposes if the original superblock gets corrupted for some reason. ext4, for example, keeps redundant copies of the superblock in each block group.

Though since littlefs's redundant superblocks can be distributed anywhere in the filesystem, I don't know how practically useful this actually is.