search

tytso / e2fsprogs

Ext2/3/4 file system utilities
http://ext4.wiki.kernel.org
373 stars 219 forks source link

e2mkdir 'No free space in the directory' failure #181

Closed Gateworks closed 5 months ago

Gateworks commented 6 months ago

I build firmware images for embdedded systems and I have a root filesystem generation process that creates an ext4 filesystem using mkfs.ext4 (mkfs.ext4 -q -F -O ^metadata_csum rootfs.ext4) and populating it by mounting it and installing files. This is done on a system with mke2fs v1.46.5.

In a follow-on process I use e2tools e2cp/e2mkdir to add files to it and I'm hitting a failure where e2mkdir errors out with 'No free space in the directory' yet as far as I can tell there is plenty of space.

I've built e2tools from the latest git source linking with e2fsprogs from its latest git source to try to better understand the failure. Before I got too deep into the details of the ext filesystem format I figured I would ask for some help here: e2mkdir fails with: No free space in the directory (EXT2_ET_DIR_NO_SPACE) ext2fs_link() fails with EXT2_ET_DIR_NO_SPACE ext2fs_dir_iterate2() does not set ls.done flag

After original creation of the ext4 I see the following details: $ e2freefrag rootfs.ext4 Device: rootfs.ext4 Blocksize: 4096 bytes Total blocks: 655360 Free blocks: 170921 (26.1%)

Min. free extent: 4 KB Max. free extent: 516064 KB Avg. free extent: 11992 KB Num. free extent: 57

HISTOGRAM OF FREE EXTENT SIZES: Extent Size Range : Free extents Free Blocks Percent 4K... 8K- : 6 6 0.00% 8K... 16K- : 7 16 0.01% 16K... 32K- : 8 37 0.02% 32K... 64K- : 16 158 0.09% 64K... 128K- : 1 30 0.02% 256K... 512K- : 4 413 0.24% 512K... 1024K- : 2 362 0.21% 1M... 2M- : 10 4293 2.51% 8M... 16M- : 1 3822 2.24% 128M... 256M- : 1 32768 19.17% 256M... 512M- : 1 129016 75.48%

$ tune2fs -l rootfs.ext4 tune2fs 1.46.5 (30-Dec-2021) Filesystem volume name: rootfs Last mounted on: /tmp/tmp.TKm2wiMNtR Filesystem UUID: 82ddb049-9636-4b26-b9cc-a0628ebfcfeb Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: user_xattr acl Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 163840 Block count: 655360 Reserved block count: 32768 Overhead clusters: 28590 Free blocks: 170921 Free inodes: 130337 First block: 0 Block size: 4096 Fragment size: 4096 Group descriptor size: 64 Reserved GDT blocks: 319 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Flex block group size: 16 Filesystem created: Thu Feb 15 14:48:14 2024 Last mount time: Thu Feb 15 14:48:14 2024 Last write time: Thu Feb 15 14:48:20 2024 Mount count: 1 Maximum mount count: -1 Last checked: Thu Feb 15 14:48:14 2024 Check interval: 0 () Lifetime writes: 1817 MB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 32 Desired extra isize: 32 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: b9544c01-e03f-40da-b209-435591957e26 Journal backup: inode blocks

After failed e2mkdir: $ e2freefrag rootfs.ext4 Device: rootfs.ext4 Blocksize: 4096 bytes Total blocks: 655360 Free blocks: 109063 (16.6%)

Min. free extent: 436252 KB Max. free extent: 436252 KB Avg. free extent: 436252 KB Num. free extent: 1

HISTOGRAM OF FREE EXTENT SIZES: Extent Size Range : Free extents Free Blocks Percent 256M... 512M- : 1 109063 100.00%

$ tune2fs -l rootfs.ext4 tune2fs 1.46.5 (30-Dec-2021) Filesystem volume name: rootfs Last mounted on: /tmp/tmp.TKm2wiMNtR Filesystem UUID: 82ddb049-9636-4b26-b9cc-a0628ebfcfeb Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: user_xattr acl Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 163840 Block count: 655360 Reserved block count: 32768 Overhead clusters: 28590 Free blocks: 109063 Free inodes: 124748 First block: 0 Block size: 4096 Fragment size: 4096 Group descriptor size: 64 Reserved GDT blocks: 319 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Flex block group size: 16 Filesystem created: Thu Feb 15 14:48:14 2024 Last mount time: Thu Feb 15 14:48:14 2024 Last write time: Fri Feb 16 09:51:53 2024 Mount count: 1 Maximum mount count: -1 Last checked: Thu Feb 15 14:48:14 2024 Check interval: 0 () Lifetime writes: 2971 MB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 32 Desired extra isize: 32 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: b9544c01-e03f-40da-b209-435591957e26 Journal backup: inode blocks

As you can see there are plenty of free blocks available but it looks like there is something funny about the 'extents' and I'm not really clear what that means. Am I running out of inodes? What could be going wrong here?

tytso commented 5 months ago

This is a bug in e2tools. The ext2fs library was originally designed for use by tools like e2fsck, so things like ext2fs_link() are a very low level library function. If ext2fs_link() fails with the error EXT2_ET_DIR_NO_SPACE the calling program needs to call ext2fs_expand_dir() and then retry the call to ext2fs_link(). For example, please see:

https://github.com/tytso/e2fsprogs/blob/260dfea450e387cbd2c8de79a7c2eeacc26f74e9/misc/fuse2fs.c#L972