'btrfs fi us' misreports freespace for raid0 with dissimilar sized devices

[cmurf]

# uname -r
5.8.0-0.rc3.20200701git7c30b859a947.1.fc33.x86_64
# btrfs --version
btrfs-progs v5.7

Summary: The central problem is that 'df' is correct, and 'btrfs fi us' Free is incorrect. User space reports out of space when I fallocate 70M, consistent with 'df', but 'btrfs fi us' information suggests this should succeed.

# df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/vda3             15G  905M   60M  94% /mnt/sysroot

# btrfs fi us /mnt/sysroot
Overall:
    Device size:          14.01GiB
    Device allocated:           1.38GiB
    Device unallocated:          12.63GiB
    Device missing:             0.00B
    Used:             901.58MiB
    Free (estimated):          12.64GiB    (min: 6.33GiB)
    Data ratio:                  1.00
    Metadata ratio:              2.00
    Global reserve:           3.25MiB    (used: 0.00B)
    Multiple profiles:                no

Data,RAID0: Size:890.00MiB, Used:870.08MiB (97.76%)
   /dev/vda3     445.00MiB
   /dev/vdb1     445.00MiB

Metadata,RAID1: Size:256.00MiB, Used:15.73MiB (6.15%)
   /dev/vda3     256.00MiB
   /dev/vdb1     256.00MiB

System,RAID1: Size:8.00MiB, Used:16.00KiB (0.20%)
   /dev/vda3       8.00MiB
   /dev/vdb1       8.00MiB

Unallocated:
   /dev/vda3      21.00MiB
   /dev/vdb1      12.61GiB
#

https://lore.kernel.org/linux-btrfs/CAJCQCtRfXy9YDqZQgYz0_djd78oEMwiYY7Og2x=J=w361FENWg@mail.gmail.com/T/#u

[cmurf]

This is arguably a difficult bug to solve, for either the Size value for 'df' or the Free value for 'btrfs fi us' - the volume is 15G but that's not usable space at all due to the lack of single stripe raid0 support. So the reporting could go either way. It's more correct for df's Size and Usage's Free to report actual usable. But...

This is made worse by #270 though, in that users are more likely to get into this weird situation in the first place. So that one should get higher priority in my opinion.

[cmurf]

Andrei argues this may be correct behavior, because this Free space is available for conversion. It's just that there are two kinds of "free" space, in this odd situation. It's not free in the current configuration for data, but it is free for conversion to another profile.

OH I just had a thought. Is the out of space actually due to metadata raid1? It can't allocate another raid1 chunk on the small device? I'm not sure whether raid0 requires two stripes minimum, or if it can fallback to single device/single chunk raid0 block groups? https://lore.kernel.org/linux-btrfs/0326afd3-9e14-b682-30e7-1c8ae44813ea@lechevalier.se/T/#t

[kdave]

Yes raid0 needs 2 stripes, the degenerate case of raid0 on single device is forbidden (volume.c: btrfs_raid_array[RAID0] .devs_min = 2)

[kdave]

The value in 'df' is correct as it takes into account the current allocation policy in kernel, the raid0 stripes cannot be spread into the remaining free space (13G) but the 'fi us' does not reflect that and reports the free space. The data ratio is calculated from existing profiles and the rest of the space is divided by that, which works for non-striped profiles. In progs git cmds/filesystem-usage.c print_filesystem_usage_overall there's a fixme to address the per-device adjustments.

[kdave]

As a data point, the degenerate raid0 will be allowed in 5.15 and fi us now reports size from df/statfs, so this may help to understand the free space situation.