E: Failed to destroy qgroup - E: Failed to remove snapshot

LaMarMot commented 8 months ago

Hello, I use timeshift with autosnap when I upgrade system, and since some time there is always an error related to qgroup and the ability to remove snapshots. Don't quite understand what happened. Could somebody please help me looking into it ?

That's console message during update:

(1/1) Creating Timeshift snapshot before upgrade...
Using system disk as snapshot device for creating snapshots in BTRFS mode
Mounted '/dev/dm-0 (nvme0n1p6)' at '/run/timeshift/3371/backup'
Creating new backup...(BTRFS)
Saving to device: /dev/dm-0, mounted at path: /run/timeshift/3371/backup
Created directory: /run/timeshift/3371/backup/timeshift-btrfs/snapshots/2024-03-13_11-21-16
Created subvolume snapshot: /run/timeshift/3371/backup/timeshift-btrfs/snapshots/2024-03-13_11-21-16/@
Created control file: /run/timeshift/3371/backup/timeshift-btrfs/snapshots/2024-03-13_11-21-16/info.json
BTRFS Snapshot saved successfully (0s)
Tagged snapshot '2024-03-13_11-21-16': ondemand
------------------------------------------------------------------------------
Maximum backups exceeded for backup level 'daily'
Mounted '/dev/dm-0 (nvme0n1p6)' at '/run/timeshift/3534/backup'
------------------------------------------------------------------------------
Removing snapshot: 2024-02-28_09-52-55
Deleted directory: /run/timeshift/3534/backup/timeshift-btrfs/snapshots/2024-02-28_09-52-55
Removed snapshot: 2024-02-28_09-52-55
------------------------------------------------------------------------------
Mounted '/dev/dm-0 (nvme0n1p6)' at '/run/timeshift/3573/backup'
------------------------------------------------------------------------------
Removing snapshot: 2024-02-28_10-46-09
Deleting subvolume: @ (Id:2335)
Deleted subvolume: @ (Id:2335)

Destroying qgroup: 0/2335
E: Failed to destroy qgroup: '0/2335'
E: Failed to remove snapshot: 2024-02-28_10-46-09
------------------------------------------------------------------------------
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-linux-zen
Found initrd image: /boot/initramfs-linux-zen.img
Found fallback initrd image(s) in /boot:  initramfs-linux-zen-fallback.img
Found linux image: /boot/vmlinuz-linux-lts
Found initrd image: /boot/initramfs-linux-lts.img
Found fallback initrd image(s) in /boot:  initramfs-linux-lts-fallback.img
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/initramfs-linux.img
Found fallback initrd image(s) in /boot:  initramfs-linux-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Found Windows Boot Manager on /dev/nvme0n1p1@/EFI/Microsoft/Boot/bootmgfw.efi
Adding boot menu entry for UEFI Firmware Settings ...
Detecting snapshots ...
Found snapshot: 2024-03-13 11:21:16 | timeshift-btrfs/snapshots/2024-03-13_11-21-16/@ | ondemand       | {timeshift-autosnap} {created before upgrade} |
Found snapshot: 2024-03-12 21:49:14 | timeshift-btrfs/snapshots/2024-03-12_21-49-14/@ | ondemand       | {timeshift-autosnap} {created before upgrade} |
[...]
Found snapshot: 2024-03-09 21:19:15 | timeshift-btrfs/snapshots/2024-03-09_21-19-15/@ | ondemand       | {timeshift-autosnap} {created before upgrade} |
[...]
Found snapshot: 2024-03-08 13:00:00 | timeshift-btrfs/snapshots/2024-03-08_13-00-00/@ | daily          | N/A                                           |
[...]
Found 28 snapshot(s)
Unmount /tmp/grub-btrfs.95vv2zuRRF .. Success
done

Attached are the logs 2024-03-13_13-00-00_backup.log 2024-03-13_12-00-00_backup.log 2024-03-13_11-29-24_ondemand.log 2024-03-13_11-27-45_gui.log 2024-03-13_11-25-37_gui.log 2024-03-13_11-21-18_delete.log 2024-03-13_11-21-17_delete.log 2024-03-13_11-21-17_list-snapshots.log 2024-03-13_11-21-16_ondemand.log

There's always the same error related to destroy qgroup

and as well regarding cron job.

I'm sorry for reporting "badly", I don't really know how to investigate Thanks lot

LaMarMot commented 8 months ago

$ sudo btrfs qgroup show .
Qgroupid    Referenced    Exclusive   Path 
--------    ----------    ---------   ---- 
0/5          200.00KiB    200.00KiB   <toplevel>
0/256          6.18GiB      1.25MiB   @
0/257        250.85GiB    250.84GiB   @home
0/258          3.06MiB      3.06MiB   @cache
0/259        319.64MiB    319.64MiB   @log
0/267          6.91GiB      6.91GiB   @var-cache-pacman-pkg
0/268         20.00GiB     20.00GiB   @swap
0/291          5.17GiB      5.16GiB   timeshift-btrfs/snapshots/2021-07-22_00-30-27/@
0/1387           0.00B        0.00B   <stale>
0/1388           0.00B        0.00B   <stale>
0/1389           0.00B        0.00B   <stale>
0/1390           0.00B        0.00B   <stale>
0/1391           0.00B        0.00B   <stale>
0/1392           0.00B        0.00B   <stale>
0/1396           0.00B        0.00B   <stale>
0/1397           0.00B        0.00B   <stale>
0/1398           0.00B        0.00B   <stale>
0/1399           0.00B        0.00B   <stale>
0/1401           0.00B        0.00B   <stale>
0/1403           0.00B        0.00B   <stale>
0/1404           0.00B        0.00B   <stale>
0/1405           0.00B        0.00B   <stale>
0/1406           0.00B        0.00B   <stale>
0/1407           0.00B        0.00B   <stale>
0/1408           0.00B        0.00B   <stale>
0/1409           0.00B        0.00B   <stale>
0/1410           0.00B        0.00B   <stale>
0/1411           0.00B        0.00B   <stale>
0/1412           0.00B        0.00B   <stale>
0/1413           0.00B        0.00B   <stale>
0/1414           0.00B        0.00B   <stale>
0/1415           0.00B        0.00B   <stale>
0/1416           0.00B        0.00B   <stale>
0/1417           0.00B        0.00B   <stale>
0/1418           0.00B        0.00B   <stale>
0/1419           0.00B        0.00B   <stale>
0/1420           0.00B        0.00B   <stale>
0/1421           0.00B        0.00B   <stale>
0/1422           0.00B        0.00B   <stale>
0/1423           0.00B        0.00B   <stale>
0/1424           0.00B        0.00B   <stale>
0/1425           0.00B        0.00B   <stale>
0/1426           0.00B        0.00B   <stale>
0/1427           0.00B        0.00B   <stale>
0/1428           0.00B        0.00B   <stale>
0/1429           0.00B        0.00B   <stale>
0/1430           0.00B        0.00B   <stale>
0/1431           0.00B        0.00B   <stale>
0/1432           0.00B        0.00B   <stale>
0/1433           0.00B        0.00B   <stale>
0/1434           0.00B        0.00B   <stale>
0/1435           0.00B        0.00B   <stale>
0/1436           0.00B        0.00B   <stale>
0/1437           0.00B        0.00B   <stale>
0/1438           0.00B        0.00B   <stale>
0/1439           0.00B        0.00B   <stale>
0/1440           0.00B        0.00B   <stale>
0/1441           0.00B        0.00B   <stale>
0/1442           0.00B        0.00B   <stale>
0/1443           0.00B        0.00B   <stale>
0/1444           0.00B        0.00B   <stale>
0/1445           0.00B        0.00B   <stale>
0/1446           0.00B        0.00B   <stale>
0/1447           0.00B        0.00B   <stale>
0/1448           0.00B        0.00B   <stale>
0/1449           0.00B        0.00B   <stale>
0/1450           0.00B        0.00B   <stale>
0/1451           0.00B        0.00B   <stale>
0/1452           0.00B        0.00B   <stale>
0/1453           0.00B        0.00B   <stale>
0/1454           0.00B        0.00B   <stale>
0/1455           0.00B        0.00B   <stale>
0/1456           0.00B        0.00B   <stale>
0/1457           0.00B        0.00B   <stale>
0/1458           0.00B        0.00B   <stale>
0/1459           0.00B        0.00B   <stale>
0/1460           0.00B        0.00B   <stale>
0/1461           0.00B        0.00B   <stale>
0/1462           0.00B        0.00B   <stale>
0/1463           0.00B        0.00B   <stale>
0/1464           0.00B        0.00B   <stale>
0/1465           0.00B        0.00B   <stale>
0/1466           0.00B        0.00B   <stale>
0/1467           0.00B        0.00B   <stale>
0/1468           0.00B        0.00B   <stale>
0/1469           0.00B        0.00B   <stale>
0/1470           0.00B        0.00B   <stale>
0/1471           0.00B        0.00B   <stale>
0/1472           0.00B        0.00B   <stale>
0/1473           0.00B        0.00B   <stale>
0/1474           0.00B        0.00B   <stale>
0/1475           0.00B        0.00B   <stale>
0/1476           0.00B        0.00B   <stale>
0/1477           0.00B        0.00B   <stale>
0/1478           0.00B        0.00B   <stale>
0/1479           0.00B        0.00B   <stale>
0/1480           0.00B        0.00B   <stale>
0/1481           0.00B        0.00B   <stale>
0/1482           0.00B        0.00B   <stale>
0/1483           0.00B        0.00B   <stale>
0/1484           0.00B        0.00B   <stale>
0/1485           0.00B        0.00B   <stale>
0/1486           0.00B        0.00B   <stale>
0/1487           0.00B        0.00B   <stale>
0/1488           0.00B        0.00B   <stale>
0/1489           0.00B        0.00B   <stale>
0/1490           0.00B        0.00B   <stale>
0/1491           0.00B        0.00B   <stale>
0/1492           0.00B        0.00B   <stale>
0/1493           0.00B        0.00B   <stale>
0/1494           0.00B        0.00B   <stale>
0/1495           0.00B        0.00B   <stale>
0/1496           0.00B        0.00B   <stale>
0/1497           0.00B        0.00B   <stale>
0/1498           0.00B        0.00B   <stale>
0/1499           0.00B        0.00B   <stale>
0/1500           0.00B        0.00B   <stale>
0/1501           0.00B        0.00B   <stale>
0/1502           0.00B        0.00B   <stale>
0/1503           0.00B        0.00B   <stale>
0/1504           0.00B        0.00B   <stale>
0/1505           0.00B        0.00B   <stale>
0/1506           0.00B        0.00B   <stale>
0/1507           0.00B        0.00B   <stale>
0/1508           0.00B        0.00B   <stale>
0/1509           0.00B        0.00B   <stale>
0/1510           0.00B        0.00B   <stale>
0/1511           0.00B        0.00B   <stale>
0/1512           0.00B        0.00B   <stale>
0/1513           0.00B        0.00B   <stale>
0/1514           0.00B        0.00B   <stale>
0/1515           0.00B        0.00B   <stale>
0/1516           0.00B        0.00B   <stale>
0/1517           0.00B        0.00B   <stale>
0/1518           0.00B        0.00B   <stale>
0/1519           0.00B        0.00B   <stale>
0/1520           0.00B        0.00B   <stale>
0/1521           0.00B        0.00B   <stale>
0/1522           0.00B        0.00B   <stale>
0/1523           0.00B        0.00B   <stale>
0/1524           0.00B        0.00B   <stale>
0/1525           0.00B        0.00B   <stale>
0/1526           0.00B        0.00B   <stale>
0/1527           0.00B        0.00B   <stale>
0/1528           0.00B        0.00B   <stale>
0/1529           0.00B        0.00B   <stale>
0/1530           0.00B        0.00B   <stale>
0/1531           0.00B        0.00B   <stale>
0/1532           0.00B        0.00B   <stale>
0/1533           0.00B        0.00B   <stale>
0/1534           0.00B        0.00B   <stale>
0/1535           0.00B        0.00B   <stale>
0/1536           0.00B        0.00B   <stale>
0/1537           0.00B        0.00B   <stale>
0/1538           0.00B        0.00B   <stale>
0/1539           0.00B        0.00B   <stale>
0/1540           0.00B        0.00B   <stale>
0/1541           0.00B        0.00B   <stale>
0/1542           0.00B        0.00B   <stale>
0/1543           0.00B        0.00B   <stale>
0/1544           0.00B        0.00B   <stale>
0/1545           0.00B        0.00B   <stale>
0/1546           0.00B        0.00B   <stale>
0/1547           0.00B        0.00B   <stale>
0/1548           0.00B        0.00B   <stale>
0/1549           0.00B        0.00B   <stale>
0/1550           0.00B        0.00B   <stale>
0/1551           0.00B        0.00B   <stale>
0/1552           0.00B        0.00B   <stale>
0/1553           0.00B        0.00B   <stale>
0/1554           0.00B        0.00B   <stale>
0/1555           0.00B        0.00B   <stale>
0/1556           0.00B        0.00B   <stale>
0/1557           0.00B        0.00B   <stale>
0/1558           0.00B        0.00B   <stale>
0/1559           0.00B        0.00B   <stale>
0/1560           0.00B        0.00B   <stale>
0/1561           0.00B        0.00B   <stale>
0/1562           0.00B        0.00B   <stale>
0/1563           0.00B        0.00B   <stale>
0/1564           0.00B        0.00B   <stale>
0/1565           0.00B        0.00B   <stale>
0/1566           0.00B        0.00B   <stale>
0/1567           0.00B        0.00B   <stale>
0/1568           0.00B        0.00B   <stale>
0/1569           0.00B        0.00B   <stale>
0/1570           0.00B        0.00B   <stale>
0/1571           0.00B        0.00B   <stale>
0/1572           0.00B        0.00B   <stale>
0/1573           0.00B        0.00B   <stale>
0/1574           0.00B        0.00B   <stale>
0/1575           0.00B        0.00B   <stale>
0/1576           0.00B        0.00B   <stale>
0/1577           0.00B        0.00B   <stale>
0/1578           0.00B        0.00B   <stale>
0/1579           0.00B        0.00B   <stale>
0/1580           0.00B        0.00B   <stale>
0/1581           0.00B        0.00B   <stale>
0/1582           0.00B        0.00B   <stale>
0/1583           0.00B        0.00B   <stale>
0/1584           0.00B        0.00B   <stale>
0/1585           0.00B        0.00B   <stale>
0/1586           0.00B        0.00B   <stale>
0/1587           0.00B        0.00B   <stale>
0/1589           0.00B        0.00B   <stale>
0/1598           0.00B        0.00B   <stale>
0/1602           0.00B        0.00B   <stale>
0/1603           0.00B        0.00B   <stale>
0/1610           0.00B        0.00B   <stale>
0/1611           0.00B        0.00B   <stale>
0/1612           0.00B        0.00B   <stale>
0/2288           0.00B        0.00B   <stale>
0/2290           0.00B        0.00B   <stale>
0/2291           0.00B        0.00B   <stale>
0/2293           0.00B        0.00B   <stale>
0/2294           0.00B        0.00B   <stale>
0/2296           0.00B        0.00B   <stale>
0/2297           0.00B        0.00B   <stale>
0/2299           0.00B        0.00B   <stale>
0/2301           0.00B        0.00B   <stale>
0/2302           0.00B        0.00B   <stale>
0/2305           0.00B        0.00B   <stale>
0/2307           0.00B        0.00B   <stale>
0/2310           0.00B        0.00B   <stale>
0/2311           0.00B        0.00B   <stale>
0/2313           0.00B        0.00B   <stale>
0/2314           0.00B        0.00B   <stale>
0/2318           0.00B        0.00B   <stale>
0/2319           0.00B        0.00B   <stale>
0/2320           0.00B        0.00B   <stale>
0/2323           0.00B        0.00B   <stale>
0/2324           0.00B        0.00B   <stale>
0/2325           0.00B        0.00B   <stale>
0/2326           0.00B        0.00B   <stale>
0/2327           0.00B        0.00B   <stale>
0/2328           0.00B        0.00B   <stale>
0/2329           0.00B        0.00B   <stale>
0/2330           0.00B        0.00B   <stale>
0/2331           0.00B        0.00B   <stale>
0/2332           0.00B        0.00B   <stale>
0/2333           0.00B        0.00B   <stale>
0/2334           0.00B        0.00B   <stale>
0/2335           0.00B        0.00B   <stale>
0/2336           0.00B        0.00B   <stale>
0/2337           0.00B        0.00B   <stale>
0/2338           0.00B        0.00B   <stale>
0/2339           0.00B        0.00B   <stale>
0/2340         6.20GiB    708.79MiB   timeshift-btrfs/snapshots/2024-03-01_12-12-43/@
0/2341           0.00B        0.00B   <stale>
0/2342         6.19GiB     10.77MiB   timeshift-btrfs/snapshots/2024-03-01_17-26-33/@
0/2343           0.00B        0.00B   <stale>
0/2344         6.19GiB      1.47MiB   timeshift-btrfs/snapshots/2024-03-02_19-08-06/@
0/2345         6.20GiB      1.52MiB   timeshift-btrfs/snapshots/2024-03-02_19-36-32/@
0/2346           0.00B        0.00B   <stale>
0/2347         6.20GiB     10.07MiB   timeshift-btrfs/snapshots/2024-03-03_12-26-51/@
0/2348         6.19GiB      9.88MiB   timeshift-btrfs/snapshots/2024-03-03_13-39-29/@
0/2349           0.00B        0.00B   <stale>
0/2350         6.24GiB     12.78MiB   timeshift-btrfs/snapshots/2024-03-03_19-29-23/@
0/2351           0.00B        0.00B   <stale>
0/2352           0.00B        0.00B   <stale>
0/2353           0.00B        0.00B   <stale>
0/2354           0.00B        0.00B   <stale>
0/2355           0.00B        0.00B   <stale>
0/2356           0.00B        0.00B   <stale>
0/2357           0.00B        0.00B   <stale>
0/2358         6.24GiB     11.44MiB   timeshift-btrfs/snapshots/2024-03-04_01-21-21/@
0/2359           0.00B        0.00B   <stale>
0/2360         6.24GiB     12.46MiB   timeshift-btrfs/snapshots/2024-03-04_13-16-55/@
0/2361           0.00B        0.00B   <stale>
0/2362           0.00B        0.00B   <stale>
0/2363           0.00B        0.00B   <stale>
0/2364           0.00B        0.00B   <stale>
0/2365           0.00B        0.00B   <stale>
0/2366           0.00B        0.00B   <stale>
0/2367         6.34GiB     20.60MiB   timeshift-btrfs/snapshots/2024-03-07_20-45-35/@
0/2368         6.35GiB      5.10MiB   timeshift-btrfs/snapshots/2024-03-07_21-38-52/@
0/2369           0.00B        0.00B   <stale>
0/2370           0.00B        0.00B   <stale>
0/2371           0.00B        0.00B   <stale>
0/2372         6.36GiB    202.92MiB   timeshift-btrfs/snapshots/2024-03-08_13-00-00/@
0/2373         6.16GiB     10.19MiB   timeshift-btrfs/snapshots/2024-03-08_17-06-31/@
0/2374         6.16GiB     12.77MiB   timeshift-btrfs/snapshots/2024-03-08_17-21-23/@
0/2375           0.00B        0.00B   <stale>
0/2376           0.00B        0.00B   <stale>
0/2377         6.16GiB     12.95MiB   timeshift-btrfs/snapshots/2024-03-08_23-25-25/@
0/2378           0.00B        0.00B   <stale>
0/2379         6.16GiB     10.12MiB   timeshift-btrfs/snapshots/2024-03-09_13-00-01/@
0/2380         6.17GiB     10.10MiB   timeshift-btrfs/snapshots/2024-03-09_21-19-15/@
0/2381         6.17GiB     10.68MiB   timeshift-btrfs/snapshots/2024-03-09_22-18-34/@
0/2382         6.17GiB     10.17MiB   timeshift-btrfs/snapshots/2024-03-10_13-00-01/@
0/2383         6.17GiB      4.32MiB   timeshift-btrfs/snapshots/2024-03-10_13-38-28/@
0/2384           0.00B        0.00B   <stale>
0/2385         6.17GiB      1.26MiB   timeshift-btrfs/snapshots/2024-03-10_14-00-00/@
0/2386         6.17GiB      1.26MiB   timeshift-btrfs/snapshots/2024-03-10_14-01-58/@
0/2387         6.17GiB     13.79MiB   timeshift-btrfs/snapshots/2024-03-11_13-00-00/@
0/2388         6.17GiB     10.09MiB   timeshift-btrfs/snapshots/2024-03-12_13-00-01/@
0/2389         6.17GiB     10.05MiB   timeshift-btrfs/snapshots/2024-03-12_21-49-14/@
0/2390         6.18GiB      2.14MiB   timeshift-btrfs/snapshots/2024-03-13_11-21-16/@
0/2391         6.18GiB      1.23MiB   timeshift-btrfs/snapshots/2024-03-13_11-29-24/@

callegar commented 8 months ago

Same here, on manjaro. Lots of "stale" qgroups reported. Disabling quota groups makes timeshift complain about that, but not erroring.

callegar commented 8 months ago

This comes with a significant difficulty in getting rid of the stale qgroups. Running btrfs qgroup clear-stale / often returns a device or resource busy error. If you wait a significant amount of time (minutes), then the operation goes through.

So I suspect that timeshift tries to remove a qgroup, fails with the "device or resource busy", errors (as in this report), and leaves the stale qgroups around...

mclang commented 7 months ago

So I can run btrfs qgroup clear-stale / and if there is Device or resource busy -errors, I just wait some and try again?

Anagastes commented 5 months ago

I can confirm this issue.

I have currently created a workaround for myself (for systems with pacman)

Very dirty, but it helps me to keep the groups clean.

If someone does it better, please let me know. I was a bit too lazy.

In /usr/share/libalpm/hooks/01-btrfs-cleanup.hook

[Trigger]
Operation = Upgrade
Type = Package
Target = *

[Action]
When = PostTransaction
Exec = /usr/bin/sh -c ' sleep 60 && btrfs qgroup clear-stale / && sudo btrfs quota rescan /'
Description = Cleanup qgroup snapshot after upgrade...

An error for ‘device is busy’ can still occurs, but it deletes old qgroups and does not keep them indefinitely...

Bildschirmfoto_20240609_171834

linuxmint / timeshift

E: Failed to destroy qgroup - E: Failed to remove snapshot #287