Zygo / bees

Best-Effort Extent-Same, a btrfs dedupe agent
GNU General Public License v3.0
625 stars 56 forks source link

How to remove? #280

Open Anonymous941 opened 2 months ago

Anonymous941 commented 2 months ago

I've run bees, and now the filesystem is reporting that there is no space, despite 1 gigabyte of free space. I assume it's the metadata of bees, but I have no idea how to remove it. How do I just remove the hash tables and other data that bees created?

kakra commented 2 months ago

You've probably run into an issue where bees first increases meta data usage on the filesystem before it can actually clean up and free space, and in that first phase, it ran into an issue with no more space left. You cannot remove this "meta data", it's not from bees, it's from the file system itself.

Try removing old snapshots if you do have some. It should clean up duplicated meta data, and may also free some partially shared extents. Then let bees continue so it can finally clean up and free actual data extents.

lilydjwg commented 2 months ago

Try removing old snapshots if you do have some.

No. Don't do this if metadata has no space left. There is a bug that may permanently put the filesystem into an errored mode (read-only).

Try to run btrfs balance start -dusage=10 mountpoint to see if you can free up some block groups from the data part. If you can't, add a temporary disk and do that balance.

Once you have enough unallocated space (at least >1GiB; run btrfs filesystem usage mountpoint and look for Device unallocated:), you can start delete snapshots to release some metadata usage (run btrfs filesystem usage mountpoint and look for Metadata,). You can also delete files. You can balance some metadata if it's too sparse.

kakra commented 2 months ago

No. Don't do this if metadata has no space left. There is a bug that may permanently put the filesystem into an errored mode (read-only).

Good point.

Zygo commented 2 months ago

The hash table is stored in .beeshome at the top level of the filesystem. The default size is 1 GiB, so deleting that should help free up one block group. Then follow the other advice--try balancing, adding space on another device, and removing unshared files and snapshots--in that order.

Anonymous941 commented 2 months ago

What ended up making my computer bootable again is running btrfs filesystem resize 230g, then btrfs filesystem resize max. I then tried to run bees again, and the same problem occurred, despite me freeing about 40/250 gigabytes. I'm not sure how to free more space, and don't want to delete all my snapshots

No. Don't do this if metadata has no space left. There is a bug that may permanently put the filesystem into an errored mode (read-only).

Try to run btrfs balance start -dusage=10 mountpoint to see if you can free up some block groups from the data part. If you can't, add a temporary disk and do that balance.

Once you have enough unallocated space (at least >1GiB; run btrfs filesystem usage mountpoint and look for Device unallocated:), you can start delete snapshots to release some metadata usage (run btrfs filesystem usage mountpoint and look for Metadata,). You can also delete files. You can balance some metadata if it's too sparse.

Should I still do this now that it's out of read-only mode? Currently it says that there's only 1.00 MiB free, despite btrfs filesystem df showing a lot of free space:

Data, single: total=228.02GiB, used=208.00GiB
System, DUP: total=8.00MiB, used=48.00KiB
Metadata, DUP: total=5.17GiB, used=4.32GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
lilydjwg commented 2 months ago

You have 20GiB free space allocated as data block groups. Run something like btrfs balance start -dusage=90 mountpoint to get them out as unallocated so that it can be allocated as metadata.

You have pretty low free space anyway. Running bees on a lot of existing snapshots will occupy a significant amount of metadata space (and it takes a lot of time). I'm afraid that you may not have enough space to finish the initial run.

Anonymous941 commented 2 months ago

It turns out that there are a lot of btrbk snapshots that I could delete. But it still says that there is only 1.00 MiB under Device unallocated, so I'm running the balance command now

Anonymous941 commented 2 months ago

Is it normal for btrfs balance start to take several minutes? I thought it just started a background process

Anonymous941 commented 2 months ago

After running, Device unallocated is now 21.00GiB. I've started bees again

Anonymous941 commented 2 months ago

It was able to complete its first run. I'm leaving this issue open as I'm not sure if it's intended behavior to put the partition in this state