Unprivileged free space info

opk12 commented 1 month ago

I would like to warn when the free space is low, in a graphical app. So my script runs unprivileged. FileSystem.usage() always requires root. Is it possible to have it work like btrfs filesystem usage, which prints the filesystem's grand totals?

$  python3
Python 3.12.6 (main, Sep  7 2024, 14:20:15) [GCC 14.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import btrfs
>>> with btrfs.FileSystem("/") as fs:
...     print(fs.usage())
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python3/dist-packages/btrfs/ctree.py", line 1061, in usage
    return btrfs.fs_usage.FsUsage(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/btrfs/fs_usage.py", line 423, in __init__
    devices = list(fs.devices())
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/btrfs/ctree.py", line 835, in devices
    for header, data in btrfs.ioctl.search_v2(self.fd, tree, min_key, max_key):
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/btrfs/ioctl.py", line 462, in _search
    fcntl.ioctl(fd, IOC_TREE_SEARCH_V2, buf)
PermissionError: [Errno 1] Operation not permitted

opk12 commented 1 month ago

My script is doing this, is there a better way, sorry I'm a newbie at btrfs

import re
import subprocess

output = subprocess.check_output("btrfs fi usage -b /".split(), stderr=subprocess.DEVNULL)
lines = [x for x in output.decode('utf-8').splitlines() if 'min:' in x]
re.match(".*min: ([0-9]+).*", lines[0]).group(1)

knorrie commented 1 month ago

Hi!

This is an interesting puzzle. Some of the btrfs kernel api functions indeed do require root level access, and some don't.

Since btrfs fi usage and python-btrfs both use the same underlying kernel functions, we can already be sure that it is possible to write a python equivalent of what btrfs fi usage is doing.

So, if you're in for a fun exercise to quickly learn more about btrfs, then what you could do is:

find out what btrfs fi usage actually is doing under the hood to give you this (min: xyz) number you're interested in
execute the same steps using Python code
end up with exact that value in bytes, without having to grep it from text output!
(and, while doing so, skip all unnecessary work that btrfs fi usage might be doing that you're not interested in)

This is exactly the type of little programs that this Python btrfs library is intended for to make.

To figure out what btrfs fi usage is actually using, you can run it using strace: strace btrfs fi usage -b /

I just did that here, the output is not that long. This will already tell you which actual kernel functions it's using. Look for the lines that look like ioctl(3, BTRFS_IOC_XXX_YYY).

In the strace output, you can see that one of the first things it tries is calling BTRFS_IOC_TREE_SEARCH which results in an -1 EPERM (Operation not permitted). Then it will print this "WARNING: cannot read detailed chunk info" and continue with the unprivileged path in the code, which uses BTRFS_IOC_FS_INFO and BTRFS_IOC_DEV_INFO and later also BTRFS_IOC_SPACE_INFO to gather whatever data it can get.

After getting some numbers, the btrfs fi usage code might do some extra calculations to determine this min: value. When looking at the code for it, you should be able to spot those: https://github.com/kdave/btrfs-progs/blob/devel/cmds/filesystem-usage.c I mean, there should be a place in the code where it prints the "Free (estimated)" and "min:" text, so from there you can for example search back what it did before.

I hope this helps as some starting tips to start playing around!

knorrie commented 1 month ago

Some extra things:

In strace output, this is where it tries to search btrfs metadata tree 3 (BTRFS_CHUNK_TREE_OBJECTID is 3, not the first 3 in the next line, that's file descriptor 3...):

ioctl(3, BTRFS_IOC_TREE_SEARCH, {key={tree_id=BTRFS_CHUNK_TREE_OBJECTID, min_objectid=0, max_objectid=UINT64_MAX, min_offset=0, max_offset=UINT64_MAX, min_transid=0, max_transid=UINT64_MAX, min_type=255, max_type=0, nr_items=4096}}) = -1 EPERM (Operation not permitted)

The shortest way to get chunk objects in a list with python-btrfs is:

>>> list(fs.chunks())
[...]
PermissionError: [Errno 1] Operation not permitted

There you get the same error!

When btrfs fi usage calls FS_INFO and DEV_INFO, it looks a bit like this:

ioctl(3, BTRFS_IOC_FS_INFO, {max_id=1, num_devices=1, fsid=a6a41c77-b726-40a5-a6cc-08cbd6abf1dd, nodesize=16384, sectorsize=4096, clone_alignment=4096, flags=0}) = 0

We can do:

>>> print(fs.fs_info())
max_id 1 num_devices 1 fsid a6a41c77-b726-40a5-a6cc-08cbd6abf1dd nodesize 16384 sectorsize 4096 clone_alignment 4096

From that info it learns that the highest device number in the filesystem is 1 (max_id=1), and then it just tries looking up info about device 0 (number 0 is normally not in use, but it can be e.g. during a device replace operation) and device 1:

ioctl(3, BTRFS_IOC_DEV_INFO, {devid=makedev(0, 0)}) = -1 ENODEV (No such device) ioctl(3, BTRFS_IOC_DEV_INFO, {devid=makedev(0, 0x1)} => {uuid=5bf9f2d0-4f80-4c0d-ac7e-e67cfa94df1a, bytes_used=143906570240, total_bytes=268435456000, path="/dev/mapper/test"}) = 0

We can do:

>>> print(fs.dev_info(0))
OSError: [Errno 19] No such device

>>> print(fs.dev_info(1))
devid 1 uuid 5bf9f2d0-4f80-4c0d-ac7e-e67cfa94df1a bytes_used 143906570240 total_bytes 268435456000 path /dev/mapper/test

Etcetera... Use the python-btrfs reference docs for info about all the different objects and their fields etc. :)

knorrie / python-btrfs

Unprivileged free space info #48