check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1

SoulInfernoDE commented 3 years ago

Hello, when i tried to backup my btrfs root partition with clonezilla it throws errors and then aborted.

I booted in a live linux live env. and installed btrfs-progs 5.11 and did a btrfs check --readonly /dev/sda6:

mint@mint:~/Downloads/btrfs-progs$ btrfs version
btrfs-progs v5.11 

mint@mint:~/Downloads/btrfs-progs$ sudo btrfs check --readonly /dev/sda6
Opening filesystem to check...
Checking filesystem on /dev/sda6
UUID: 7677e01a-7637-43f3-b9aa-598e065e24ad
[1/7] checking root items
[2/7] checking extents
WARNING: tree block [12090130432, 12090146816) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12090195968, 12090212352) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12090261504, 12090277888) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12090785792, 12090802176) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12090851328, 12090867712) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12090916864, 12090933248) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12090982400, 12090998784) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12091768832, 12091785216) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12092293120, 12092309504) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12092358656, 12092375040) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12092489728, 12092506112) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12092620800, 12092637184) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12092751872, 12092768256) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12093014016, 12093030400) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12093145088, 12093161472) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12093210624, 12093227008) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12093276160, 12093292544) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12093341696, 12093358080) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12095045632, 12095062016) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12095635456, 12095651840) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12096225280, 12096241664) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12096421888, 12096438272) crosses 64K page boudnary, may cause problem for 64K page system
WARNING: tree block [12096552960, 12096569344) crosses 64K page boudnary, may cause problem for 64K page system

...

WARNING: tree block [34911076352, 34911092736) crosses 64K page boudnary, may cause problem for 64K page system
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
checksum verify failed on 34688204800 found 000000B6 wanted 00000000
checksum verify failed on 34688204800 found 000000B6 wanted 00000000
bad tree block 34688204800, bytenr mismatch, want=34688204800, have=0
checksum verify failed on 35269099520 found 000000B6 wanted 00000000
checksum verify failed on 35269099520 found 000000B6 wanted 00000000
bad tree block 35269099520, bytenr mismatch, want=35269099520, have=0
checksum verify failed on 35269115904 found 000000B6 wanted 00000000
checksum verify failed on 35269115904 found 000000B6 wanted 00000000
bad tree block 35269115904, bytenr mismatch, want=35269115904, have=0
checksum verify failed on 35269148672 found 000000B6 wanted 00000000
checksum verify failed on 35269148672 found 000000B6 wanted 00000000
bad tree block 35269148672, bytenr mismatch, want=35269148672, have=0
checksum verify failed on 35269165056 found 000000B6 wanted 00000000
checksum verify failed on 35269165056 found 000000B6 wanted 00000000
bad tree block 35269165056, bytenr mismatch, want=35269165056, have=0
checksum verify failed on 12019974144 found 000000B6 wanted 00000000
checksum verify failed on 12019974144 found 000000B6 wanted 00000000
bad tree block 12019974144, bytenr mismatch, want=12019974144, have=0
checksum verify failed on 34578956288 found 000000B6 wanted 00000000
checksum verify failed on 34578956288 found 000000B6 wanted 00000000
bad tree block 34578956288, bytenr mismatch, want=34578956288, have=0
checksum verify failed on 12384960512 found 000000B6 wanted 00000000
checksum verify failed on 12384960512 found 000000B6 wanted 00000000
bad tree block 12384960512, bytenr mismatch, want=12384960512, have=0
checksum verify failed on 35218259968 found 000000B6 wanted 00000000
checksum verify failed on 35218259968 found 000000B6 wanted 00000000
bad tree block 35218259968, bytenr mismatch, want=35218259968, have=0
checksum verify failed on 35218440192 found 000000B6 wanted 00000000
checksum verify failed on 35218440192 found 000000B6 wanted 00000000
bad tree block 35218440192, bytenr mismatch, want=35218440192, have=0
checksum verify failed on 34583478272 found 000000B6 wanted 00000000
checksum verify failed on 34583478272 found 000000B6 wanted 00000000
bad tree block 34583478272, bytenr mismatch, want=34583478272, have=0
checksum verify failed on 34589851648 found 000000B6 wanted 00000000
checksum verify failed on 34589851648 found 000000B6 wanted 00000000
bad tree block 34589851648, bytenr mismatch, want=34589851648, have=0
checksum verify failed on 34593275904 found 000000B6 wanted 00000000
checksum verify failed on 34593275904 found 000000B6 wanted 00000000
bad tree block 34593275904, bytenr mismatch, want=34593275904, have=0
checksum verify failed on 34594684928 found 000000B6 wanted 00000000
checksum verify failed on 34594684928 found 000000B6 wanted 00000000
bad tree block 34594684928, bytenr mismatch, want=34594684928, have=0
checksum verify failed on 34597093376 found 000000B6 wanted 00000000
checksum verify failed on 34597093376 found 000000B6 wanted 00000000
bad tree block 34597093376, bytenr mismatch, want=34597093376, have=0
checksum verify failed on 35219963904 found 000000B6 wanted 00000000
checksum verify failed on 35219963904 found 000000B6 wanted 00000000
bad tree block 35219963904, bytenr mismatch, want=35219963904, have=0
checksum verify failed on 34618359808 found 000000B6 wanted 00000000
checksum verify failed on 34618359808 found 000000B6 wanted 00000000
bad tree block 34618359808, bytenr mismatch, want=34618359808, have=0
checksum verify failed on 34624602112 found 000000B6 wanted 00000000
checksum verify failed on 34624602112 found 000000B6 wanted 00000000
bad tree block 34624602112, bytenr mismatch, want=34624602112, have=0
checksum verify failed on 34631827456 found 000000B6 wanted 00000000
checksum verify failed on 34631827456 found 000000B6 wanted 00000000
bad tree block 34631827456, bytenr mismatch, want=34631827456, have=0
checksum verify failed on 34648457216 found 000000B6 wanted 00000000
checksum verify failed on 34648457216 found 000000B6 wanted 00000000
bad tree block 34648457216, bytenr mismatch, want=34648457216, have=0
checksum verify failed on 12400918528 found 000000B6 wanted 00000000
checksum verify failed on 12400918528 found 000000B6 wanted 00000000
bad tree block 12400918528, bytenr mismatch, want=12400918528, have=0
checksum verify failed on 12404375552 found 000000B6 wanted 00000000
checksum verify failed on 12404375552 found 000000B6 wanted 00000000
bad tree block 12404375552, bytenr mismatch, want=12404375552, have=0
checksum verify failed on 12427902976 found 000000B6 wanted 00000000
checksum verify failed on 12427902976 found 000000B6 wanted 00000000
bad tree block 12427902976, bytenr mismatch, want=12427902976, have=0
checksum verify failed on 35212427264 found 000000B6 wanted 00000000
checksum verify failed on 35212427264 found 000000B6 wanted 00000000
bad tree block 35212427264, bytenr mismatch, want=35212427264, have=0
checksum verify failed on 62652416 found 000000B6 wanted 00000000
checksum verify failed on 62652416 found 000000B6 wanted 00000000
bad tree block 62652416, bytenr mismatch, want=62652416, have=0
checksum verify failed on 35219718144 found 000000B6 wanted 00000000
checksum verify failed on 35219718144 found 000000B6 wanted 00000000
bad tree block 35219718144, bytenr mismatch, want=35219718144, have=0
checksum verify failed on 35219144704 found 000000B6 wanted 00000000
checksum verify failed on 35219144704 found 000000B6 wanted 00000000
bad tree block 35219144704, bytenr mismatch, want=35219144704, have=0
checksum verify failed on 62898176 found 000000B6 wanted 00000000
checksum verify failed on 62898176 found 000000B6 wanted 00000000
bad tree block 62898176, bytenr mismatch, want=62898176, have=0
checksum verify failed on 35225878528 found 000000B6 wanted 00000000
checksum verify failed on 35225878528 found 000000B6 wanted 00000000
bad tree block 35225878528, bytenr mismatch, want=35225878528, have=0
checksum verify failed on 35227451392 found 000000B6 wanted 00000000
checksum verify failed on 35227451392 found 000000B6 wanted 00000000
bad tree block 35227451392, bytenr mismatch, want=35227451392, have=0
checksum verify failed on 63504384 found 000000B6 wanted 00000000
checksum verify failed on 63504384 found 000000B6 wanted 00000000
bad tree block 63504384, bytenr mismatch, want=63504384, have=0
checksum verify failed on 34581889024 found 000000B6 wanted 00000000
checksum verify failed on 34581889024 found 000000B6 wanted 00000000
bad tree block 34581889024, bytenr mismatch, want=34581889024, have=0
checksum verify failed on 35219996672 found 000000B6 wanted 00000000
checksum verify failed on 35219996672 found 000000B6 wanted 00000000
bad tree block 35219996672, bytenr mismatch, want=35219996672, have=0
checksum verify failed on 34611576832 found 000000B6 wanted 00000000
checksum verify failed on 34611576832 found 000000B6 wanted 00000000
bad tree block 34611576832, bytenr mismatch, want=34611576832, have=0
checksum verify failed on 600489984 found 000000B6 wanted 00000000
checksum verify failed on 600489984 found 000000B6 wanted 00000000
bad tree block 600489984, bytenr mismatch, want=600489984, have=0
checksum verify failed on 600129536 found 000000B6 wanted 00000000
checksum verify failed on 600129536 found 000000B6 wanted 00000000
bad tree block 600129536, bytenr mismatch, want=600129536, have=0
checksum verify failed on 600244224 found 000000B6 wanted 00000000
checksum verify failed on 600244224 found 000000B6 wanted 00000000
bad tree block 600244224, bytenr mismatch, want=600244224, have=0
checksum verify failed on 599998464 found 000000B6 wanted 00000000
checksum verify failed on 599998464 found 000000B6 wanted 00000000
bad tree block 599998464, bytenr mismatch, want=599998464, have=0
checksum verify failed on 567279616 found 000000B6 wanted 00000000
checksum verify failed on 567279616 found 000000B6 wanted 00000000
bad tree block 567279616, bytenr mismatch, want=567279616, have=0
checksum verify failed on 568164352 found 000000B6 wanted 00000000
checksum verify failed on 568164352 found 000000B6 wanted 00000000
bad tree block 568164352, bytenr mismatch, want=568164352, have=0
checksum verify failed on 34615132160 found 000000B6 wanted 00000000
checksum verify failed on 34615132160 found 000000B6 wanted 00000000
bad tree block 34615132160, bytenr mismatch, want=34615132160, have=0
checksum verify failed on 34618605568 found 000000B6 wanted 00000000
checksum verify failed on 34618605568 found 000000B6 wanted 00000000
bad tree block 34618605568, bytenr mismatch, want=34618605568, have=0
checksum verify failed on 34622799872 found 000000B6 wanted 00000000
checksum verify failed on 34622799872 found 000000B6 wanted 00000000
bad tree block 34622799872, bytenr mismatch, want=34622799872, have=0
checksum verify failed on 34635218944 found 000000B6 wanted 00000000
checksum verify failed on 34635218944 found 000000B6 wanted 00000000
bad tree block 34635218944, bytenr mismatch, want=34635218944, have=0
checksum verify failed on 35211952128 found 000000B6 wanted 00000000
checksum verify failed on 35211952128 found 000000B6 wanted 00000000
bad tree block 35211952128, bytenr mismatch, want=35211952128, have=0
checksum verify failed on 35280273408 found 000000B6 wanted 00000000
checksum verify failed on 35280273408 found 000000B6 wanted 00000000
bad tree block 35280273408, bytenr mismatch, want=35280273408, have=0
checksum verify failed on 35211739136 found 000000B6 wanted 00000000
checksum verify failed on 35211739136 found 000000B6 wanted 00000000
bad tree block 35211739136, bytenr mismatch, want=35211739136, have=0
checksum verify failed on 12379783168 found 000000B6 wanted 00000000
checksum verify failed on 12379783168 found 000000B6 wanted 00000000
bad tree block 12379783168, bytenr mismatch, want=12379783168, have=0
checksum verify failed on 34647883776 found 000000B6 wanted 00000000
checksum verify failed on 34647883776 found 000000B6 wanted 00000000
bad tree block 34647883776, bytenr mismatch, want=34647883776, have=0
checksum verify failed on 49184768 found 000000B6 wanted 00000000
checksum verify failed on 49184768 found 000000B6 wanted 00000000
bad tree block 49184768, bytenr mismatch, want=49184768, have=0
checksum verify failed on 34688204800 found 000000B6 wanted 00000000
checksum verify failed on 34688204800 found 000000B6 wanted 00000000
bad tree block 34688204800, bytenr mismatch, want=34688204800, have=0
checksum verify failed on 35269099520 found 000000B6 wanted 00000000
checksum verify failed on 35269099520 found 000000B6 wanted 00000000
bad tree block 35269099520, bytenr mismatch, want=35269099520, have=0
checksum verify failed on 35269115904 found 000000B6 wanted 00000000
checksum verify failed on 35269115904 found 000000B6 wanted 00000000
bad tree block 35269115904, bytenr mismatch, want=35269115904, have=0
checksum verify failed on 35269148672 found 000000B6 wanted 00000000
checksum verify failed on 35269148672 found 000000B6 wanted 00000000
bad tree block 35269148672, bytenr mismatch, want=35269148672, have=0
checksum verify failed on 35269165056 found 000000B6 wanted 00000000
checksum verify failed on 35269165056 found 000000B6 wanted 00000000
bad tree block 35269165056, bytenr mismatch, want=35269165056, have=0
check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1
btrfs(+0x1bb4e)[0x55f2036c4b4e]
btrfs(qgroup_verify_all+0x4c6)[0x55f2036c6323]
btrfs(+0x4af9e)[0x55f2036f3f9e]
btrfs(main+0x98)[0x55f2036c0ee8]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f2ec135e0b3]
btrfs(_start+0x2e)[0x55f2036c0a6e]
Abgebrochen
mint@mint:~/Downloads/btrfs-progs$

I really need to copy this partition to my new external ssd from my internal ssd. note: the system behaves normal i didn't noticed any bad behaviour while using normal. I just ran into this when i tried to copy the btrfs partition to the new ssd.

Linux live kernel version im using:

Linux mint 5.8.0-45-generic #51~20.04.1-Ubuntu SMP Tue Feb 23 13:46:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

In my prod. laptop system i am using kernel 5.11.7

best regards SI

SoulInfernoDE commented 3 years ago

Now i have done a btrfs check and repair with the new updated version: 5.11.1


mint@mint:~/Downloads/btrfs-progs$ sudo btrfs check --repair /dev/sda6
enabling repair mode
WARNING:

    Do not use --repair unless you are advised to do so by a developer
    or an experienced user, and then only after having accepted that no
    fsck can successfully repair all types of filesystem corruption. Eg.
    some software or hardware bugs can fatally damage a volume.
    The operation will start in 10 seconds.
    Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting repair.
Opening filesystem to check...
Checking filesystem on /dev/sda6
UUID: 7677e01a-7637-43f3-b9aa-598e065e24ad
[1/7] checking root items
Fixed 0 roots.
[2/7] checking extents
No device size related problem found
[3/7] checking free space cache
cache and super generation don't match, space cache will be invalidated
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
checksum verify failed on 34688204800 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34688204800 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34688204800, bytenr mismatch, want=34688204800, have=0
checksum verify failed on 35269099520 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35269099520 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35269099520, bytenr mismatch, want=35269099520, have=0
checksum verify failed on 35269115904 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35269115904 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35269115904, bytenr mismatch, want=35269115904, have=0
checksum verify failed on 35269148672 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35269148672 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35269148672, bytenr mismatch, want=35269148672, have=0
checksum verify failed on 35269165056 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35269165056 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35269165056, bytenr mismatch, want=35269165056, have=0
checksum verify failed on 12019974144 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 12019974144 wanted 0x00000000 found 0xb6bde3e4
bad tree block 12019974144, bytenr mismatch, want=12019974144, have=0
checksum verify failed on 34578956288 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34578956288 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34578956288, bytenr mismatch, want=34578956288, have=0
checksum verify failed on 12384960512 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 12384960512 wanted 0x00000000 found 0xb6bde3e4
bad tree block 12384960512, bytenr mismatch, want=12384960512, have=0
checksum verify failed on 35218259968 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35218259968 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35218259968, bytenr mismatch, want=35218259968, have=0
checksum verify failed on 35218440192 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35218440192 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35218440192, bytenr mismatch, want=35218440192, have=0
checksum verify failed on 34583478272 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34583478272 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34583478272, bytenr mismatch, want=34583478272, have=0
checksum verify failed on 34589851648 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34589851648 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34589851648, bytenr mismatch, want=34589851648, have=0
checksum verify failed on 34593275904 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34593275904 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34593275904, bytenr mismatch, want=34593275904, have=0
checksum verify failed on 34594684928 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34594684928 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34594684928, bytenr mismatch, want=34594684928, have=0
checksum verify failed on 34597093376 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34597093376 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34597093376, bytenr mismatch, want=34597093376, have=0
checksum verify failed on 35219963904 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35219963904 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35219963904, bytenr mismatch, want=35219963904, have=0
checksum verify failed on 34618359808 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34618359808 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34618359808, bytenr mismatch, want=34618359808, have=0
checksum verify failed on 34624602112 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34624602112 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34624602112, bytenr mismatch, want=34624602112, have=0
checksum verify failed on 34631827456 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34631827456 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34631827456, bytenr mismatch, want=34631827456, have=0
checksum verify failed on 34648457216 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34648457216 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34648457216, bytenr mismatch, want=34648457216, have=0
checksum verify failed on 12400918528 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 12400918528 wanted 0x00000000 found 0xb6bde3e4
bad tree block 12400918528, bytenr mismatch, want=12400918528, have=0
checksum verify failed on 12404375552 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 12404375552 wanted 0x00000000 found 0xb6bde3e4
bad tree block 12404375552, bytenr mismatch, want=12404375552, have=0
checksum verify failed on 12427902976 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 12427902976 wanted 0x00000000 found 0xb6bde3e4
bad tree block 12427902976, bytenr mismatch, want=12427902976, have=0
checksum verify failed on 35212427264 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35212427264 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35212427264, bytenr mismatch, want=35212427264, have=0
checksum verify failed on 62652416 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 62652416 wanted 0x00000000 found 0xb6bde3e4
bad tree block 62652416, bytenr mismatch, want=62652416, have=0
checksum verify failed on 35219718144 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35219718144 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35219718144, bytenr mismatch, want=35219718144, have=0
checksum verify failed on 35219144704 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35219144704 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35219144704, bytenr mismatch, want=35219144704, have=0
checksum verify failed on 62898176 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 62898176 wanted 0x00000000 found 0xb6bde3e4
bad tree block 62898176, bytenr mismatch, want=62898176, have=0
checksum verify failed on 35225878528 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35225878528 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35225878528, bytenr mismatch, want=35225878528, have=0
checksum verify failed on 35227451392 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35227451392 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35227451392, bytenr mismatch, want=35227451392, have=0
checksum verify failed on 63504384 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 63504384 wanted 0x00000000 found 0xb6bde3e4
bad tree block 63504384, bytenr mismatch, want=63504384, have=0
checksum verify failed on 34581889024 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34581889024 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34581889024, bytenr mismatch, want=34581889024, have=0
checksum verify failed on 35219996672 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35219996672 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35219996672, bytenr mismatch, want=35219996672, have=0
checksum verify failed on 34611576832 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34611576832 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34611576832, bytenr mismatch, want=34611576832, have=0
checksum verify failed on 600489984 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 600489984 wanted 0x00000000 found 0xb6bde3e4
bad tree block 600489984, bytenr mismatch, want=600489984, have=0
checksum verify failed on 600129536 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 600129536 wanted 0x00000000 found 0xb6bde3e4
bad tree block 600129536, bytenr mismatch, want=600129536, have=0
checksum verify failed on 600244224 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 600244224 wanted 0x00000000 found 0xb6bde3e4
bad tree block 600244224, bytenr mismatch, want=600244224, have=0
checksum verify failed on 599998464 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 599998464 wanted 0x00000000 found 0xb6bde3e4
bad tree block 599998464, bytenr mismatch, want=599998464, have=0
checksum verify failed on 567279616 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 567279616 wanted 0x00000000 found 0xb6bde3e4
bad tree block 567279616, bytenr mismatch, want=567279616, have=0
checksum verify failed on 568164352 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 568164352 wanted 0x00000000 found 0xb6bde3e4
bad tree block 568164352, bytenr mismatch, want=568164352, have=0
checksum verify failed on 34615132160 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34615132160 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34615132160, bytenr mismatch, want=34615132160, have=0
checksum verify failed on 34618605568 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34618605568 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34618605568, bytenr mismatch, want=34618605568, have=0
checksum verify failed on 34622799872 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34622799872 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34622799872, bytenr mismatch, want=34622799872, have=0
checksum verify failed on 34635218944 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34635218944 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34635218944, bytenr mismatch, want=34635218944, have=0
checksum verify failed on 35211952128 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35211952128 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35211952128, bytenr mismatch, want=35211952128, have=0
checksum verify failed on 35280273408 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35280273408 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35280273408, bytenr mismatch, want=35280273408, have=0
checksum verify failed on 35211739136 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35211739136 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35211739136, bytenr mismatch, want=35211739136, have=0
checksum verify failed on 12379783168 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 12379783168 wanted 0x00000000 found 0xb6bde3e4
bad tree block 12379783168, bytenr mismatch, want=12379783168, have=0
checksum verify failed on 34647883776 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34647883776 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34647883776, bytenr mismatch, want=34647883776, have=0
checksum verify failed on 49184768 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 49184768 wanted 0x00000000 found 0xb6bde3e4
bad tree block 49184768, bytenr mismatch, want=49184768, have=0
checksum verify failed on 34688204800 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 34688204800 wanted 0x00000000 found 0xb6bde3e4
bad tree block 34688204800, bytenr mismatch, want=34688204800, have=0
checksum verify failed on 35269099520 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35269099520 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35269099520, bytenr mismatch, want=35269099520, have=0
checksum verify failed on 35269115904 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35269115904 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35269115904, bytenr mismatch, want=35269115904, have=0
checksum verify failed on 35269148672 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35269148672 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35269148672, bytenr mismatch, want=35269148672, have=0
checksum verify failed on 35269165056 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 35269165056 wanted 0x00000000 found 0xb6bde3e4
bad tree block 35269165056, bytenr mismatch, want=35269165056, have=0
check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1
btrfs(+0x1bb6e)[0x5597bd9c6b6e]
btrfs(qgroup_verify_all+0x4c6)[0x5597bd9c8343]
btrfs(+0x4b2e7)[0x5597bd9f62e7]
btrfs(main+0x98)[0x5597bd9c2f08]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f0d9b7180b3]
btrfs(_start+0x2e)[0x5597bd9c2a8e]
Abgebrochen
mint@mint:~/Downloads/btrfs-progs$

It seems like the BUG is still there. The brtfs check doesn't complete it stops..

adam900710 commented 3 years ago

The warning WARNING: tree block [34911076352, 34911092736) crosses 64K page boudnary, may cause problem for 64K page system is a false alert and fixed in latest btrfs-progs release.

The checksum error means some tree blocks are zeroed out.

I guess it's your quota root cleared but by somehow there is still an quota root item in your root tree?

Please provide the following output:

 btrfs ins dump-tree -t quota /dev/sda6
 btrfs ins dump-tree -t root /dev/sda6

SoulInfernoDE commented 3 years ago

Okay, thank you for assisting :-)

here is the output:

btrfs ins dump-tree -t quota /dev/sda6


mint@mint:~/Downloads/btrfsprogsgit/btrfs-progs$ sudo btrfs ins dump-tree -t quota /dev/sda6
btrfs-progs v5.11.1 
quota tree key (QUOTA_TREE ROOT_ITEM 0) 
leaf 12847890432 items 19 free space 15216 generation 445512 owner QUOTA_TREE
leaf 12847890432 flags 0x1(WRITTEN) backref revision 1
fs uuid 7677e01a-7637-43f3-b9aa-598e065e24ad
chunk uuid 49255341-7f3c-4180-adb3-944527737422
    item 0 key (0 QGROUP_STATUS 0) itemoff 16251 itemsize 32
        version 1 generation 445512 flags ON|INCONSISTENT scan 0
    item 1 key (0 QGROUP_INFO 0/5) itemoff 16211 itemsize 40
        generation 445361
        referenced 20480 referenced_compressed 20480
        exclusive 20480 exclusive_compressed 20480
    item 2 key (0 QGROUP_INFO 0/257) itemoff 16171 itemsize 40
        generation 445511
        referenced 44348018688 referenced_compressed 44348018688
        exclusive 208789504 exclusive_compressed 208789504
    item 3 key (0 QGROUP_INFO 0/671) itemoff 16131 itemsize 40
        generation 444697
        referenced 16384 referenced_compressed 16384
        exclusive 16384 exclusive_compressed 16384
    item 4 key (0 QGROUP_INFO 0/672) itemoff 16091 itemsize 40
        generation 444697
        referenced 16384 referenced_compressed 16384
        exclusive 16384 exclusive_compressed 16384
    item 5 key (0 QGROUP_INFO 0/729) itemoff 16051 itemsize 40
        generation 445509
        referenced 44269760512 referenced_compressed 44269760512
        exclusive 130531328 exclusive_compressed 130531328
    item 6 key (0 QGROUP_INFO 255/671) itemoff 16011 itemsize 40
        generation 444697
        referenced 16384 referenced_compressed 16384
        exclusive 16384 exclusive_compressed 16384
    item 7 key (0 QGROUP_INFO 255/672) itemoff 15971 itemsize 40
        generation 444697
        referenced 16384 referenced_compressed 16384
        exclusive 16384 exclusive_compressed 16384
    item 8 key (0 QGROUP_LIMIT 0/5) itemoff 15931 itemsize 40
        flags 0
        max_referenced 0 max_exclusive 0
        rsv_referenced 0 rsv_exclusive 0
    item 9 key (0 QGROUP_LIMIT 0/257) itemoff 15891 itemsize 40
        flags 0
        max_referenced 0 max_exclusive 0
        rsv_referenced 0 rsv_exclusive 0
    item 10 key (0 QGROUP_LIMIT 0/671) itemoff 15851 itemsize 40
        flags 0
        max_referenced 0 max_exclusive 0
        rsv_referenced 0 rsv_exclusive 0
    item 11 key (0 QGROUP_LIMIT 0/672) itemoff 15811 itemsize 40
        flags 0
        max_referenced 0 max_exclusive 0
        rsv_referenced 0 rsv_exclusive 0
    item 12 key (0 QGROUP_LIMIT 0/729) itemoff 15771 itemsize 40
        flags 0
        max_referenced 0 max_exclusive 0
        rsv_referenced 0 rsv_exclusive 0
    item 13 key (0 QGROUP_LIMIT 255/671) itemoff 15731 itemsize 40
        flags 0
        max_referenced 0 max_exclusive 0
        rsv_referenced 0 rsv_exclusive 0
    item 14 key (0 QGROUP_LIMIT 255/672) itemoff 15691 itemsize 40
        flags 0
        max_referenced 0 max_exclusive 0
        rsv_referenced 0 rsv_exclusive 0
    item 15 key (0/671 QGROUP_RELATION 255/671) itemoff 15691 itemsize 0
    item 16 key (0/672 QGROUP_RELATION 255/672) itemoff 15691 itemsize 0
    item 17 key (255/671 QGROUP_RELATION 0/671) itemoff 15691 itemsize 0
    item 18 key (255/672 QGROUP_RELATION 0/672) itemoff 15691 itemsize 0
total bytes 120307318784
bytes used 64917647360
uuid 7677e01a-7637-43f3-b9aa-598e065e24ad

btrfs ins dump-tree -t root /dev/sda6

mint@mint:~/Downloads/btrfsprogsgit/btrfs-progs$ sudo btrfs ins dump-tree -t root /dev/sda6
btrfs-progs v5.11.1 
root tree
node 12846252032 level 1 items 2 free space 491 generation 445512 owner ROOT_TREE
node 12846252032 flags 0x1(WRITTEN) backref revision 1
fs uuid 7677e01a-7637-43f3-b9aa-598e065e24ad
chunk uuid 49255341-7f3c-4180-adb3-944527737422
    key (EXTENT_TREE ROOT_ITEM 0) block 12848136192 gen 445512
    key (629 INODE_ITEM 0) block 12848742400 gen 445512
leaf 12848136192 items 101 free space 299 generation 445512 owner ROOT_TREE
leaf 12848136192 flags 0x1(WRITTEN) backref revision 1
fs uuid 7677e01a-7637-43f3-b9aa-598e065e24ad
chunk uuid 49255341-7f3c-4180-adb3-944527737422
    item 0 key (EXTENT_TREE ROOT_ITEM 0) itemoff 15844 itemsize 439
        generation 445512 root_dirid 0 bytenr 12847824896 level 2 refs 1
        lastsnap 0 byte_limit 0 bytes_used 57933824 flags 0x0(none)
        uuid 00000000-0000-0000-0000-000000000000
        drop key (0 UNKNOWN.0 0) level 0
    item 1 key (DEV_TREE ROOT_ITEM 0) itemoff 15405 itemsize 439
        generation 444701 root_dirid 0 bytenr 27656192 level 0 refs 1
        lastsnap 0 byte_limit 0 bytes_used 16384 flags 0x0(none)
        uuid 00000000-0000-0000-0000-000000000000
        drop key (0 UNKNOWN.0 0) level 0
    item 2 key (FS_TREE INODE_REF 6) itemoff 15388 itemsize 17
        index 0 namelen 7 name: default
    item 3 key (FS_TREE ROOT_ITEM 0) itemoff 14949 itemsize 439
        generation 445361 root_dirid 256 bytenr 12119638016 level 0 refs 1
        lastsnap 0 byte_limit 0 bytes_used 16384 flags 0x0(none)
        uuid 00000000-0000-0000-0000-000000000000
        ctransid 445361 otransid 0 stransid 0 rtransid 0
        ctime 1616653838.654192876 (2021-03-25 07:30:38)
        drop key (0 UNKNOWN.0 0) level 0
    item 4 key (FS_TREE ROOT_REF 257) itemoff 14930 itemsize 19
        root ref key dirid 256 sequence 2 name @
    item 5 key (FS_TREE ROOT_REF 729) itemoff 14911 itemsize 19
        root ref key dirid 215919 sequence 2 name @
    item 6 key (ROOT_TREE_DIR INODE_ITEM 0) itemoff 14751 itemsize 160
        generation 3 transid 0 size 0 nbytes 16384
        block group 0 mode 40755 links 1 uid 0 gid 0 rdev 0
        sequence 0 flags 0x0(none)
        atime 1539878625.0 (2018-10-18 18:03:45)
        ctime 1539878625.0 (2018-10-18 18:03:45)
        mtime 1539878625.0 (2018-10-18 18:03:45)
        otime 1539878625.0 (2018-10-18 18:03:45)
    item 7 key (ROOT_TREE_DIR INODE_REF 6) itemoff 14739 itemsize 12
        index 0 namelen 2 name: ..
    item 8 key (ROOT_TREE_DIR DIR_ITEM 2378154706) itemoff 14702 itemsize 37
        location key (FS_TREE ROOT_ITEM 18446744073709551615) type DIR
        transid 0 data_len 0 name_len 7
        name: default
    item 9 key (CSUM_TREE ROOT_ITEM 0) itemoff 14263 itemsize 439
        generation 445512 root_dirid 0 bytenr 12848185344 level 2 refs 1
        lastsnap 0 byte_limit 0 bytes_used 69451776 flags 0x0(none)
        uuid 00000000-0000-0000-0000-000000000000
        drop key (0 UNKNOWN.0 0) level 0
    item 10 key (QUOTA_TREE ROOT_ITEM 0) itemoff 13824 itemsize 439
        generation 445512 root_dirid 0 bytenr 12847890432 level 0 refs 1
        lastsnap 0 byte_limit 0 bytes_used 16384 flags 0x0(none)
        uuid 8b3d530d-b5a5-5740-ae41-1569233009f8
        drop key (0 UNKNOWN.0 0) level 0
    item 11 key (UUID_TREE ROOT_ITEM 0) itemoff 13385 itemsize 439
        generation 445355 root_dirid 0 bytenr 12089851904 level 0 refs 1
        lastsnap 0 byte_limit 0 bytes_used 16384 flags 0x0(none)
        uuid d7fc528e-aad8-b947-aa3d-eb55bbe49b87
        drop key (0 UNKNOWN.0 0) level 0
    item 12 key (256 INODE_ITEM 0) itemoff 13225 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 70023118848
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 267117 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.34217898 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 13 key (256 EXTENT_DATA 0) itemoff 13172 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 21640138752 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 14 key (257 ROOT_ITEM 0) itemoff 12733 itemsize 439
        generation 445511 root_dirid 256 bytenr 12847841280 level 2 refs 1
        lastsnap 445355 byte_limit 0 bytes_used 590348288 flags 0x0(none)
        uuid b4dc7e78-ea7e-dc43-8bd7-0580bb8e7b33
        ctransid 445511 otransid 8 stransid 0 rtransid 0
        ctime 1616655353.234792012 (2021-03-25 07:55:53)
        otime 1539878627.301248469 (2018-10-18 18:03:47)
        drop key (0 UNKNOWN.0 0) level 0
    item 15 key (257 ROOT_BACKREF 5) itemoff 12714 itemsize 19
        root backref key dirid 256 sequence 2 name @
    item 16 key (257 ROOT_REF 671) itemoff 12687 itemsize 27
        root ref key dirid 241723 sequence 106 name portables
    item 17 key (257 ROOT_REF 672) itemoff 12661 itemsize 26
        root ref key dirid 241723 sequence 107 name machines
    item 18 key (258 INODE_ITEM 0) itemoff 12501 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 58292174848
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 222367 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.30217834 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 19 key (258 EXTENT_DATA 0) itemoff 12448 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 16989659136 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 20 key (259 INODE_ITEM 0) itemoff 12288 itemsize 160
        generation 445309 transid 445309 size 262144 nbytes 23351787520
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 89080 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616653475.906271219 (2021-03-25 07:24:35)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 21 key (259 EXTENT_DATA 0) itemoff 12235 itemsize 53
        generation 445309 type 1 (regular)
        extent data disk byte 14619107328 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 22 key (260 INODE_ITEM 0) itemoff 12075 itemsize 160
        generation 445378 transid 445378 size 262144 nbytes 10988290048
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 41917 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654368.829105205 (2021-03-25 07:39:28)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 23 key (260 EXTENT_DATA 0) itemoff 12022 itemsize 53
        generation 445378 type 1 (regular)
        extent data disk byte 15915974656 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 24 key (261 INODE_ITEM 0) itemoff 11862 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 9939451904
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 37916 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.30217834 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 25 key (261 EXTENT_DATA 0) itemoff 11809 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 18039377920 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 26 key (262 INODE_ITEM 0) itemoff 11649 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 5006688256
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 19099 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.30217834 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 27 key (262 EXTENT_DATA 0) itemoff 11596 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 18387922944 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 28 key (263 INODE_ITEM 0) itemoff 11436 itemsize 160
        generation 445247 transid 445247 size 262144 nbytes 11548753920
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 44055 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616653181.466524270 (2021-03-25 07:19:41)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 29 key (263 EXTENT_DATA 0) itemoff 11383 itemsize 53
        generation 445247 type 1 (regular)
        extent data disk byte 18032308224 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 30 key (266 INODE_ITEM 0) itemoff 11223 itemsize 160
        generation 445512 transid 445512 size 262144 nbytes 13401587712
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 51123 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.446224492 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 31 key (266 EXTENT_DATA 0) itemoff 11170 itemsize 53
        generation 445512 type 1 (regular)
        extent data disk byte 18378219520 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 32 key (267 INODE_ITEM 0) itemoff 11010 itemsize 160
        generation 445399 transid 445399 size 262144 nbytes 11700797440
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 44635 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654551.658353587 (2021-03-25 07:42:31)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 33 key (267 EXTENT_DATA 0) itemoff 10957 itemsize 53
        generation 445399 type 1 (regular)
        extent data disk byte 17935257600 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 34 key (268 INODE_ITEM 0) itemoff 10797 itemsize 160
        generation 445169 transid 445169 size 262144 nbytes 13410500608
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 51157 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652675.140556397 (2021-03-25 07:11:15)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 35 key (268 EXTENT_DATA 0) itemoff 10744 itemsize 53
        generation 445169 type 1 (regular)
        extent data disk byte 18069864448 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 36 key (270 INODE_ITEM 0) itemoff 10584 itemsize 160
        generation 445399 transid 445399 size 262144 nbytes 12242386944
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 46701 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654551.658353587 (2021-03-25 07:42:31)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 37 key (270 EXTENT_DATA 0) itemoff 10531 itemsize 53
        generation 445399 type 1 (regular)
        extent data disk byte 18014924800 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 38 key (271 INODE_ITEM 0) itemoff 10371 itemsize 160
        generation 445512 transid 445512 size 262144 nbytes 39737622528
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 151587 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.446224492 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 39 key (271 EXTENT_DATA 0) itemoff 10318 itemsize 53
        generation 445512 type 1 (regular)
        extent data disk byte 8270966784 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 40 key (274 INODE_ITEM 0) itemoff 10158 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 7077101568
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 26997 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.30217834 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 41 key (274 EXTENT_DATA 0) itemoff 10105 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 19848486912 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 42 key (278 INODE_ITEM 0) itemoff 9945 itemsize 160
        generation 445404 transid 445404 size 262144 nbytes 13132890112
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 50098 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654551.954361321 (2021-03-25 07:42:31)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 43 key (278 EXTENT_DATA 0) itemoff 9892 itemsize 53
        generation 445404 type 1 (regular)
        extent data disk byte 18113155072 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 44 key (290 INODE_ITEM 0) itemoff 9732 itemsize 160
        generation 445404 transid 445404 size 262144 nbytes 15157690368
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 57822 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654551.954361321 (2021-03-25 07:42:31)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 45 key (290 EXTENT_DATA 0) itemoff 9679 itemsize 53
        generation 445404 type 1 (regular)
        extent data disk byte 18118365184 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 46 key (293 INODE_ITEM 0) itemoff 9519 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 18550095872
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 70763 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.38217962 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 47 key (293 EXTENT_DATA 0) itemoff 9466 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 21650923520 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 48 key (303 INODE_ITEM 0) itemoff 9306 itemsize 160
        generation 445512 transid 445512 size 262144 nbytes 11839471616
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 45164 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.446224492 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 49 key (303 EXTENT_DATA 0) itemoff 9253 itemsize 53
        generation 445512 type 1 (regular)
        extent data disk byte 19352846336 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 50 key (309 INODE_ITEM 0) itemoff 9093 itemsize 160
        generation 445512 transid 445512 size 262144 nbytes 5627183104
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 21466 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.450224556 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 51 key (309 EXTENT_DATA 0) itemoff 9040 itemsize 53
        generation 445512 type 1 (regular)
        extent data disk byte 19859714048 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 52 key (310 INODE_ITEM 0) itemoff 8880 itemsize 160
        generation 445512 transid 445512 size 262144 nbytes 6252134400
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 23850 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.446224492 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 53 key (310 EXTENT_DATA 0) itemoff 8827 itemsize 53
        generation 445512 type 1 (regular)
        extent data disk byte 18118627328 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 54 key (311 INODE_ITEM 0) itemoff 8667 itemsize 160
        generation 445512 transid 445512 size 262144 nbytes 2500329472
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 9538 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.450224556 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 55 key (311 EXTENT_DATA 0) itemoff 8614 itemsize 53
        generation 445512 type 1 (regular)
        extent data disk byte 19376074752 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 56 key (315 INODE_ITEM 0) itemoff 8454 itemsize 160
        generation 445510 transid 445510 size 262144 nbytes 3456368640
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 13185 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.218220843 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 57 key (315 EXTENT_DATA 0) itemoff 8401 itemsize 53
        generation 445510 type 1 (regular)
        extent data disk byte 19881472000 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 58 key (316 INODE_ITEM 0) itemoff 8241 itemsize 160
        generation 445477 transid 445477 size 262144 nbytes 4142399488
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 15802 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654562.786642760 (2021-03-25 07:42:42)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 59 key (316 EXTENT_DATA 0) itemoff 8188 itemsize 53
        generation 445477 type 1 (regular)
        extent data disk byte 19877224448 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 60 key (320 INODE_ITEM 0) itemoff 8028 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 2154037248
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 8217 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.30217834 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 61 key (320 EXTENT_DATA 0) itemoff 7975 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 21122809856 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 62 key (323 INODE_ITEM 0) itemoff 7815 itemsize 160
        generation 445510 transid 445510 size 262144 nbytes 1696071680
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 6470 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.214220779 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 63 key (323 EXTENT_DATA 0) itemoff 7762 itemsize 53
        generation 445510 type 1 (regular)
        extent data disk byte 18064728064 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 64 key (325 INODE_ITEM 0) itemoff 7602 itemsize 160
        generation 445437 transid 445437 size 262144 nbytes 1000865792
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 3818 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654554.34415591 (2021-03-25 07:42:34)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 65 key (325 EXTENT_DATA 0) itemoff 7549 itemsize 53
        generation 445437 type 1 (regular)
        extent data disk byte 19853365248 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 66 key (330 INODE_ITEM 0) itemoff 7389 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 2099773440
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 8010 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.30217834 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 67 key (330 EXTENT_DATA 0) itemoff 7336 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 21172592640 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 68 key (333 INODE_ITEM 0) itemoff 7176 itemsize 160
        generation 445068 transid 445068 size 262144 nbytes 3037200384
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 11586 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652629.529424894 (2021-03-25 07:10:29)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 69 key (333 EXTENT_DATA 0) itemoff 7123 itemsize 53
        generation 445068 type 1 (regular)
        extent data disk byte 21406212096 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 70 key (335 INODE_ITEM 0) itemoff 6963 itemsize 160
        generation 444131 transid 444131 size 262144 nbytes 2848980992
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 10868 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616276649.720499648 (2021-03-20 22:44:09)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 71 key (335 EXTENT_DATA 0) itemoff 6910 itemsize 53
        generation 444131 type 1 (regular)
        extent data disk byte 13355663360 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 72 key (341 INODE_ITEM 0) itemoff 6750 itemsize 160
        generation 444917 transid 444917 size 262144 nbytes 1639710720
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 6255 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652464.454964034 (2021-03-25 07:07:44)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 73 key (341 EXTENT_DATA 0) itemoff 6697 itemsize 53
        generation 444917 type 1 (regular)
        extent data disk byte 15932489728 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 74 key (349 INODE_ITEM 0) itemoff 6537 itemsize 160
        generation 444516 transid 444516 size 262144 nbytes 1931739136
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 7369 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616478116.787182125 (2021-03-23 06:41:56)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 75 key (349 EXTENT_DATA 0) itemoff 6484 itemsize 53
        generation 444516 type 1 (regular)
        extent data disk byte 14078214144 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 76 key (352 INODE_ITEM 0) itemoff 6324 itemsize 160
        generation 445243 transid 445243 size 262144 nbytes 2268332032
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 8653 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616653122.841826848 (2021-03-25 07:18:42)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 77 key (352 EXTENT_DATA 0) itemoff 6271 itemsize 53
        generation 445243 type 1 (regular)
        extent data disk byte 17897824256 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 78 key (365 INODE_ITEM 0) itemoff 6111 itemsize 160
        generation 444971 transid 444971 size 262144 nbytes 944242688
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 3602 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652472.402009841 (2021-03-25 07:07:52)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 79 key (365 EXTENT_DATA 0) itemoff 6058 itemsize 53
        generation 444971 type 1 (regular)
        extent data disk byte 14141972480 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 80 key (374 INODE_ITEM 0) itemoff 5898 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 18285330432
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 69753 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.30217834 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 81 key (374 EXTENT_DATA 0) itemoff 5845 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 21424533504 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 82 key (507 INODE_ITEM 0) itemoff 5685 itemsize 160
        generation 445074 transid 445074 size 262144 nbytes 1622147072
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 6188 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652629.821416796 (2021-03-25 07:10:29)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 83 key (507 EXTENT_DATA 0) itemoff 5632 itemsize 53
        generation 445074 type 1 (regular)
        extent data disk byte 21195112448 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 84 key (538 INODE_ITEM 0) itemoff 5472 itemsize 160
        generation 444655 transid 444655 size 262144 nbytes 432275456
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 1649 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616478631.126426758 (2021-03-23 06:50:31)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 85 key (538 EXTENT_DATA 0) itemoff 5419 itemsize 53
        generation 444655 type 1 (regular)
        extent data disk byte 14276055040 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 86 key (547 INODE_ITEM 0) itemoff 5259 itemsize 160
        generation 445482 transid 445482 size 262144 nbytes 802160640
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 3060 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654594.119440895 (2021-03-25 07:43:14)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 87 key (547 EXTENT_DATA 0) itemoff 5206 itemsize 53
        generation 445482 type 1 (regular)
        extent data disk byte 21642125312 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 88 key (548 INODE_ITEM 0) itemoff 5046 itemsize 160
        generation 444950 transid 444950 size 262144 nbytes 904658944
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 3451 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652471.446119426 (2021-03-25 07:07:51)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 89 key (548 EXTENT_DATA 0) itemoff 4993 itemsize 53
        generation 444950 type 1 (regular)
        extent data disk byte 16164974592 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 90 key (550 INODE_ITEM 0) itemoff 4833 itemsize 160
        generation 444950 transid 444950 size 262144 nbytes 611057664
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 2331 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652471.446119426 (2021-03-25 07:07:51)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 91 key (550 EXTENT_DATA 0) itemoff 4780 itemsize 53
        generation 444950 type 1 (regular)
        extent data disk byte 16167440384 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 92 key (614 ROOT_ITEM 354785) itemoff 4341 itemsize 439
        generation 421785 root_dirid 256 bytenr 12019712000 level 2 refs 0
        lastsnap 354785 byte_limit 0 bytes_used 452640768 flags 0x1000000000000(none)
        uuid 892239b6-4ee0-8b48-84a9-c5c866d491da
        parent_uuid b4dc7e78-ea7e-dc43-8bd7-0580bb8e7b33
        ctransid 421785 otransid 354785 stransid 0 rtransid 0
        ctime 1614722401.974267437 (2021-03-02 23:00:01)
        otime 1594031065.53396765 (2020-07-06 12:24:25)
        drop key (0 UNKNOWN.0 0) level 0
    item 93 key (616 ROOT_ITEM 355576) itemoff 3902 itemsize 439
        generation 421785 root_dirid 256 bytenr 12019761152 level 2 refs 0
        lastsnap 355576 byte_limit 0 bytes_used 453181440 flags 0x1000000000000(none)
        uuid 8de4b1e9-4b8a-8343-90ac-b0e45c29aecd
        parent_uuid b4dc7e78-ea7e-dc43-8bd7-0580bb8e7b33
        ctransid 421785 otransid 355576 stransid 0 rtransid 0
        ctime 1614722401.976164686 (2021-03-02 23:00:01)
        otime 1594391776.52862479 (2020-07-10 16:36:16)
        drop key (0 UNKNOWN.0 0) level 0
    item 94 key (617 ROOT_ITEM 355614) itemoff 3463 itemsize 439
        generation 421785 root_dirid 256 bytenr 12019957760 level 2 refs 0
        lastsnap 355614 byte_limit 0 bytes_used 454721536 flags 0x1000000000000(none)
        uuid c3256576-d4cf-2249-9a65-62df8bee6f59
        parent_uuid b4dc7e78-ea7e-dc43-8bd7-0580bb8e7b33
        ctransid 421785 otransid 355614 stransid 0 rtransid 0
        ctime 1614722401.977885503 (2021-03-02 23:00:01)
        otime 1594392677.269749085 (2020-07-10 16:51:17)
        drop key (20592873 INODE_REF 20592870) level 1
    item 95 key (620 INODE_ITEM 0) itemoff 3303 itemsize 160
        generation 445169 transid 445169 size 262144 nbytes 1399062528
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 5337 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652675.144557269 (2021-03-25 07:11:15)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 96 key (620 EXTENT_DATA 0) itemoff 3250 itemsize 53
        generation 445169 type 1 (regular)
        extent data disk byte 18388447232 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 97 key (622 INODE_ITEM 0) itemoff 3090 itemsize 160
        generation 445074 transid 445074 size 262144 nbytes 1454374912
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 5548 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652629.821416796 (2021-03-25 07:10:29)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 98 key (622 EXTENT_DATA 0) itemoff 3037 itemsize 53
        generation 445074 type 1 (regular)
        extent data disk byte 21436710912 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 99 key (625 INODE_ITEM 0) itemoff 2877 itemsize 160
        generation 445172 transid 445172 size 262144 nbytes 336855040
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 1285 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652714.43927407 (2021-03-25 07:11:54)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 100 key (625 EXTENT_DATA 0) itemoff 2824 itemsize 53
        generation 445172 type 1 (regular)
        extent data disk byte 17973768192 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
leaf 12848742400 items 138 free space 1704 generation 445512 owner ROOT_TREE
leaf 12848742400 flags 0x1(WRITTEN) backref revision 1
fs uuid 7677e01a-7637-43f3-b9aa-598e065e24ad
chunk uuid 49255341-7f3c-4180-adb3-944527737422
    item 0 key (629 INODE_ITEM 0) itemoff 16123 itemsize 160
        generation 445482 transid 445482 size 262144 nbytes 1308622848
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 4992 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654594.119440895 (2021-03-25 07:43:14)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 1 key (629 EXTENT_DATA 0) itemoff 16070 itemsize 53
        generation 445482 type 1 (regular)
        extent data disk byte 21651558400 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 2 key (637 INODE_ITEM 0) itemoff 15910 itemsize 160
        generation 445169 transid 445169 size 262144 nbytes 1001127936
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 3819 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652675.144557269 (2021-03-25 07:11:15)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 3 key (637 EXTENT_DATA 0) itemoff 15857 itemsize 53
        generation 445169 type 1 (regular)
        extent data disk byte 18388709376 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 4 key (641 INODE_ITEM 0) itemoff 15697 itemsize 160
        generation 445483 transid 445483 size 262144 nbytes 619708416
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 2364 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616654626.840250305 (2021-03-25 07:43:46)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 5 key (641 EXTENT_DATA 0) itemoff 15644 itemsize 53
        generation 445483 type 1 (regular)
        extent data disk byte 17989152768 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 6 key (643 INODE_ITEM 0) itemoff 15484 itemsize 160
        generation 444130 transid 444130 size 262144 nbytes 392167424
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 1496 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616276612.518326839 (2021-03-20 22:43:32)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 7 key (643 EXTENT_DATA 0) itemoff 15431 itemsize 53
        generation 444130 type 1 (regular)
        extent data disk byte 32402251776 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 8 key (666 INODE_ITEM 0) itemoff 15271 itemsize 160
        generation 444917 transid 444917 size 262144 nbytes 121372672
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 463 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652464.458963529 (2021-03-25 07:07:44)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 9 key (666 EXTENT_DATA 0) itemoff 15218 itemsize 53
        generation 444917 type 1 (regular)
        extent data disk byte 16119881728 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 10 key (671 ROOT_ITEM 0) itemoff 14779 itemsize 439
        generation 422215 root_dirid 256 bytenr 840761344 level 0 refs 1
        lastsnap 0 byte_limit 0 bytes_used 16384 flags 0x0(none)
        uuid 11396ed6-114f-1444-8c8b-f18779d5b08a
        ctransid 422215 otransid 397807 stransid 0 rtransid 0
        ctime 1614757110.697056167 (2021-03-03 08:38:30)
        otime 1606515655.444683014 (2020-11-27 23:20:55)
        drop key (0 UNKNOWN.0 0) level 0
    item 11 key (671 ROOT_BACKREF 257) itemoff 14752 itemsize 27
        root backref key dirid 241723 sequence 106 name portables
    item 12 key (672 ROOT_ITEM 0) itemoff 14313 itemsize 439
        generation 422215 root_dirid 256 bytenr 840777728 level 0 refs 1
        lastsnap 0 byte_limit 0 bytes_used 16384 flags 0x0(none)
        uuid f3cff87c-518f-d949-8d74-ab8e09a24f25
        ctransid 422215 otransid 397808 stransid 0 rtransid 0
        ctime 1614757110.697085938 (2021-03-03 08:38:30)
        otime 1606515655.480683420 (2020-11-27 23:20:55)
        drop key (0 UNKNOWN.0 0) level 0
    item 13 key (672 ROOT_BACKREF 257) itemoff 14287 itemsize 26
        root backref key dirid 241723 sequence 107 name machines
    item 14 key (680 INODE_ITEM 0) itemoff 14127 itemsize 160
        generation 445273 transid 445273 size 262144 nbytes 612630528
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 2337 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616653378.260003789 (2021-03-25 07:22:58)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 15 key (680 EXTENT_DATA 0) itemoff 14074 itemsize 53
        generation 445273 type 1 (regular)
        extent data disk byte 17925320704 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 16 key (682 INODE_ITEM 0) itemoff 13914 itemsize 160
        generation 445272 transid 445272 size 262144 nbytes 164888576
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 629 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616653378.228002963 (2021-03-25 07:22:58)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 17 key (682 EXTENT_DATA 0) itemoff 13861 itemsize 53
        generation 445272 type 1 (regular)
        extent data disk byte 18115731456 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 18 key (687 INODE_ITEM 0) itemoff 13701 itemsize 160
        generation 444639 transid 444639 size 262144 nbytes 94633984
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 361 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616478628.210443731 (2021-03-23 06:50:28)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 19 key (687 EXTENT_DATA 0) itemoff 13648 itemsize 53
        generation 444639 type 1 (regular)
        extent data disk byte 6368649216 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 20 key (688 INODE_ITEM 0) itemoff 13488 itemsize 160
        generation 444655 transid 444655 size 262144 nbytes 420478976
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 1604 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616478631.126426758 (2021-03-23 06:50:31)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 21 key (688 EXTENT_DATA 0) itemoff 13435 itemsize 53
        generation 444655 type 1 (regular)
        extent data disk byte 14618845184 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 22 key (704 INODE_ITEM 0) itemoff 13275 itemsize 160
        generation 434508 transid 434508 size 262144 nbytes 2359296
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 9 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1615837405.393735995 (2021-03-15 20:43:25)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 23 key (704 EXTENT_DATA 0) itemoff 13222 itemsize 53
        generation 434508 type 1 (regular)
        extent data disk byte 67358654464 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 24 key (705 INODE_ITEM 0) itemoff 13062 itemsize 160
        generation 434508 transid 434508 size 262144 nbytes 1572864
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 6 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1615837405.393735995 (2021-03-15 20:43:25)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 25 key (705 EXTENT_DATA 0) itemoff 13009 itemsize 53
        generation 434508 type 1 (regular)
        extent data disk byte 67358916608 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 26 key (706 INODE_ITEM 0) itemoff 12849 itemsize 160
        generation 444955 transid 444955 size 262144 nbytes 25690112
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 98 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652471.626098680 (2021-03-25 07:07:51)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 27 key (706 EXTENT_DATA 0) itemoff 12796 itemsize 53
        generation 444955 type 1 (regular)
        extent data disk byte 14193643520 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 28 key (707 INODE_ITEM 0) itemoff 12636 itemsize 160
        generation 444131 transid 444131 size 262144 nbytes 36175872
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 138 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616276649.736502302 (2021-03-20 22:44:09)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 29 key (707 EXTENT_DATA 0) itemoff 12583 itemsize 53
        generation 444131 type 1 (regular)
        extent data disk byte 14178959360 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 30 key (708 INODE_ITEM 0) itemoff 12423 itemsize 160
        generation 445267 transid 445267 size 262144 nbytes 473432064
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 1806 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616653378.99999660 (2021-03-25 07:22:58)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 31 key (708 EXTENT_DATA 0) itemoff 12370 itemsize 53
        generation 445267 type 1 (regular)
        extent data disk byte 18117345280 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 32 key (709 INODE_ITEM 0) itemoff 12210 itemsize 160
        generation 444971 transid 444971 size 262144 nbytes 111149056
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 424 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652472.402009841 (2021-03-25 07:07:52)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 33 key (709 EXTENT_DATA 0) itemoff 12157 itemsize 53
        generation 444971 type 1 (regular)
        extent data disk byte 14172631040 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 34 key (710 INODE_ITEM 0) itemoff 11997 itemsize 160
        generation 444950 transid 444950 size 262144 nbytes 146538496
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 559 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652471.446119426 (2021-03-25 07:07:51)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 35 key (710 EXTENT_DATA 0) itemoff 11944 itemsize 53
        generation 444950 type 1 (regular)
        extent data disk byte 16179519488 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 36 key (712 INODE_ITEM 0) itemoff 11784 itemsize 160
        generation 444917 transid 444917 size 262144 nbytes 166199296
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 634 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616652464.458963529 (2021-03-25 07:07:44)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 37 key (712 EXTENT_DATA 0) itemoff 11731 itemsize 53
        generation 444917 type 1 (regular)
        extent data disk byte 16146571264 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 38 key (715 INODE_ITEM 0) itemoff 11571 itemsize 160
        generation 444688 transid 444688 size 262144 nbytes 2883584
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 11 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616479005.759539536 (2021-03-23 06:56:45)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 39 key (715 EXTENT_DATA 0) itemoff 11518 itemsize 53
        generation 444688 type 1 (regular)
        extent data disk byte 6659932160 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 40 key (716 INODE_ITEM 0) itemoff 11358 itemsize 160
        generation 444688 transid 444688 size 262144 nbytes 6291456
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 24 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616479005.759539536 (2021-03-23 06:56:45)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 41 key (716 EXTENT_DATA 0) itemoff 11305 itemsize 53
        generation 444688 type 1 (regular)
        extent data disk byte 6673395712 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 42 key (717 INODE_ITEM 0) itemoff 11145 itemsize 160
        generation 437482 transid 437482 size 262144 nbytes 524288
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 2 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616078589.486315165 (2021-03-18 15:43:09)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 43 key (717 EXTENT_DATA 0) itemoff 11092 itemsize 53
        generation 437482 type 1 (regular)
        extent data disk byte 78395379712 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 44 key (718 INODE_ITEM 0) itemoff 10932 itemsize 160
        generation 437493 transid 437493 size 262144 nbytes 55836672
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 213 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616078607.19337539 (2021-03-18 15:43:27)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 45 key (718 EXTENT_DATA 0) itemoff 10879 itemsize 53
        generation 437493 type 1 (regular)
        extent data disk byte 78702891008 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 46 key (719 INODE_ITEM 0) itemoff 10719 itemsize 160
        generation 437482 transid 437482 size 262144 nbytes 524288
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 2 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616078589.486315165 (2021-03-18 15:43:09)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 47 key (719 EXTENT_DATA 0) itemoff 10666 itemsize 53
        generation 437482 type 1 (regular)
        extent data disk byte 78395904000 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 48 key (720 INODE_ITEM 0) itemoff 10506 itemsize 160
        generation 444688 transid 444688 size 262144 nbytes 108265472
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 413 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616479005.759539536 (2021-03-23 06:56:45)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 49 key (720 EXTENT_DATA 0) itemoff 10453 itemsize 53
        generation 444688 type 1 (regular)
        extent data disk byte 6673657856 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 50 key (721 INODE_ITEM 0) itemoff 10293 itemsize 160
        generation 444688 transid 444688 size 262144 nbytes 174850048
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 667 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616479005.763539654 (2021-03-23 06:56:45)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 51 key (721 EXTENT_DATA 0) itemoff 10240 itemsize 53
        generation 444688 type 1 (regular)
        extent data disk byte 8544157696 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 52 key (722 INODE_ITEM 0) itemoff 10080 itemsize 160
        generation 445508 transid 445508 size 262144 nbytes 763101184
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 2911 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655349.102154910 (2021-03-25 07:55:49)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 53 key (722 EXTENT_DATA 0) itemoff 10027 itemsize 53
        generation 445508 type 1 (regular)
        extent data disk byte 21611388928 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 54 key (723 INODE_ITEM 0) itemoff 9867 itemsize 160
        generation 445509 transid 445509 size 262144 nbytes 852754432
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 3253 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616655353.34217898 (2021-03-25 07:55:53)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 55 key (723 EXTENT_DATA 0) itemoff 9814 itemsize 53
        generation 445509 type 1 (regular)
        extent data disk byte 21603864576 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 56 key (724 INODE_ITEM 0) itemoff 9654 itemsize 160
        generation 444162 transid 444162 size 262144 nbytes 463470592
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 1768 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616324897.484401233 (2021-03-21 12:08:17)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 57 key (724 EXTENT_DATA 0) itemoff 9601 itemsize 53
        generation 444162 type 1 (regular)
        extent data disk byte 15947128832 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 58 key (725 INODE_ITEM 0) itemoff 9441 itemsize 160
        generation 444688 transid 444688 size 262144 nbytes 195297280
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 745 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616479005.767539772 (2021-03-23 06:56:45)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 59 key (725 EXTENT_DATA 0) itemoff 9388 itemsize 53
        generation 444688 type 1 (regular)
        extent data disk byte 13923995648 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 60 key (727 INODE_ITEM 0) itemoff 9228 itemsize 160
        generation 444688 transid 444688 size 262144 nbytes 149159936
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 569 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616479005.767539772 (2021-03-23 06:56:45)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 61 key (727 EXTENT_DATA 0) itemoff 9175 itemsize 53
        generation 444688 type 1 (regular)
        extent data disk byte 14034788352 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 62 key (728 INODE_ITEM 0) itemoff 9015 itemsize 160
        generation 444688 transid 444688 size 262144 nbytes 363331584
        block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
        sequence 1386 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
        atime 0.0 (1970-01-01 01:00:00)
        ctime 1616479005.767539772 (2021-03-23 06:56:45)
        mtime 0.0 (1970-01-01 01:00:00)
        otime 0.0 (1970-01-01 01:00:00)
    item 63 key (728 EXTENT_DATA 0) itemoff 8962 itemsize 53
        generation 444688 type 1 (regular)
        extent data disk byte 14141227008 nr 262144
        extent data offset 0 nr 262144 ram 262144
        extent compression 0 (none)
    item 64 key (729 ROOT_ITEM 445355) itemoff 8523 itemsize 439
        generation 445356 root_dirid 256 bytenr 12073222144 level 2 refs 1
        lastsnap 445355 byte_limit 0 bytes_used 589594624 flags 0x0(none)
        uuid ca796606-dbaa-094f-b444-d90559ebe48e
        parent_uuid b4dc7e78-ea7e-dc43-8bd7-0580bb8e7b33
        ctransid 445355 otransid 445355 stransid 0 rtransid 0
        ctime 1616653708.770413408 (2021-03-25 07:28:28)
        otime 1616653708.774303816 (2021-03-25 07:28:28)
        drop key (0 UNKNOWN.0 0) level 0
    item 65 key (729 ROOT_BACKREF 5) itemoff 8504 itemsize 19
        root backref key dirid 215919 sequence 2 name @
    item 66 key (FREE_SPACE UNTYPED 22020096) itemoff 8463 itemsize 41
        location key (256 INODE_ITEM 0)
        cache generation 445509 entries 449 bitmaps 8
    item 67 key (FREE_SPACE UNTYPED 1095761920) itemoff 8422 itemsize 41
        location key (258 INODE_ITEM 0)
        cache generation 445509 entries 1 bitmaps 0
    item 68 key (FREE_SPACE UNTYPED 2169503744) itemoff 8381 itemsize 41
        location key (259 INODE_ITEM 0)
        cache generation 445309 entries 0 bitmaps 0
    item 69 key (FREE_SPACE UNTYPED 3243245568) itemoff 8340 itemsize 41
        location key (260 INODE_ITEM 0)
        cache generation 445378 entries 0 bitmaps 0
    item 70 key (FREE_SPACE UNTYPED 4316987392) itemoff 8299 itemsize 41
        location key (261 INODE_ITEM 0)
        cache generation 445509 entries 0 bitmaps 0
    item 71 key (FREE_SPACE UNTYPED 5390729216) itemoff 8258 itemsize 41
        location key (262 INODE_ITEM 0)
        cache generation 445509 entries 3 bitmaps 0
    item 72 key (FREE_SPACE UNTYPED 6464471040) itemoff 8217 itemsize 41
        location key (263 INODE_ITEM 0)
        cache generation 445247 entries 0 bitmaps 0
    item 73 key (FREE_SPACE UNTYPED 7538212864) itemoff 8176 itemsize 41
        location key (266 INODE_ITEM 0)
        cache generation 445512 entries 0 bitmaps 0
    item 74 key (FREE_SPACE UNTYPED 8611954688) itemoff 8135 itemsize 41
        location key (267 INODE_ITEM 0)
        cache generation 445399 entries 0 bitmaps 0
    item 75 key (FREE_SPACE UNTYPED 9685696512) itemoff 8094 itemsize 41
        location key (268 INODE_ITEM 0)
        cache generation 445169 entries 0 bitmaps 0
    item 76 key (FREE_SPACE UNTYPED 10759438336) itemoff 8053 itemsize 41
        location key (270 INODE_ITEM 0)
        cache generation 445399 entries 0 bitmaps 0
    item 77 key (FREE_SPACE UNTYPED 11833180160) itemoff 8012 itemsize 41
        location key (271 INODE_ITEM 0)
        cache generation 445512 entries 430 bitmaps 8
    item 78 key (FREE_SPACE UNTYPED 12906921984) itemoff 7971 itemsize 41
        location key (274 INODE_ITEM 0)
        cache generation 445509 entries 1 bitmaps 0
    item 79 key (FREE_SPACE UNTYPED 13980663808) itemoff 7930 itemsize 41
        location key (278 INODE_ITEM 0)
        cache generation 445404 entries 0 bitmaps 0
    item 80 key (FREE_SPACE UNTYPED 15054405632) itemoff 7889 itemsize 41
        location key (290 INODE_ITEM 0)
        cache generation 445404 entries 0 bitmaps 0
    item 81 key (FREE_SPACE UNTYPED 16128147456) itemoff 7848 itemsize 41
        location key (293 INODE_ITEM 0)
        cache generation 445509 entries 1 bitmaps 0
    item 82 key (FREE_SPACE UNTYPED 17201889280) itemoff 7807 itemsize 41
        location key (303 INODE_ITEM 0)
        cache generation 445512 entries 4 bitmaps 0
    item 83 key (FREE_SPACE UNTYPED 18275631104) itemoff 7766 itemsize 41
        location key (309 INODE_ITEM 0)
        cache generation 445512 entries 1 bitmaps 0
    item 84 key (FREE_SPACE UNTYPED 19349372928) itemoff 7725 itemsize 41
        location key (310 INODE_ITEM 0)
        cache generation 445512 entries 1 bitmaps 0
    item 85 key (FREE_SPACE UNTYPED 20423114752) itemoff 7684 itemsize 41
        location key (311 INODE_ITEM 0)
        cache generation 445512 entries 96 bitmaps 3
    item 86 key (FREE_SPACE UNTYPED 21496856576) itemoff 7643 itemsize 41
        location key (315 INODE_ITEM 0)
        cache generation 445510 entries 256 bitmaps 7
    item 87 key (FREE_SPACE UNTYPED 22570598400) itemoff 7602 itemsize 41
        location key (316 INODE_ITEM 0)
        cache generation 445477 entries 250 bitmaps 7
    item 88 key (FREE_SPACE UNTYPED 23644340224) itemoff 7561 itemsize 41
        location key (320 INODE_ITEM 0)
        cache generation 445509 entries 121 bitmaps 0
    item 89 key (FREE_SPACE UNTYPED 24718082048) itemoff 7520 itemsize 41
        location key (323 INODE_ITEM 0)
        cache generation 445510 entries 103 bitmaps 1
    item 90 key (FREE_SPACE UNTYPED 25791823872) itemoff 7479 itemsize 41
        location key (325 INODE_ITEM 0)
        cache generation 445437 entries 165 bitmaps 3
    item 91 key (FREE_SPACE UNTYPED 26865565696) itemoff 7438 itemsize 41
        location key (330 INODE_ITEM 0)
        cache generation 445509 entries 211 bitmaps 6
    item 92 key (FREE_SPACE UNTYPED 27939307520) itemoff 7397 itemsize 41
        location key (333 INODE_ITEM 0)
        cache generation 445068 entries 287 bitmaps 7
    item 93 key (FREE_SPACE UNTYPED 29013049344) itemoff 7356 itemsize 41
        location key (335 INODE_ITEM 0)
        cache generation 444131 entries 177 bitmaps 0
    item 94 key (FREE_SPACE UNTYPED 30086791168) itemoff 7315 itemsize 41
        location key (341 INODE_ITEM 0)
        cache generation 444917 entries 125 bitmaps 1
    item 95 key (FREE_SPACE UNTYPED 31160532992) itemoff 7274 itemsize 41
        location key (349 INODE_ITEM 0)
        cache generation 444516 entries 75 bitmaps 0
    item 96 key (FREE_SPACE UNTYPED 32234274816) itemoff 7233 itemsize 41
        location key (352 INODE_ITEM 0)
        cache generation 445243 entries 213 bitmaps 5
    item 97 key (FREE_SPACE UNTYPED 33308016640) itemoff 7192 itemsize 41
        location key (365 INODE_ITEM 0)
        cache generation 444971 entries 124 bitmaps 1
    item 98 key (FREE_SPACE UNTYPED 34381758464) itemoff 7151 itemsize 41
        location key (374 INODE_ITEM 0)
        cache generation 445509 entries 519 bitmaps 8
    item 99 key (FREE_SPACE UNTYPED 35455500288) itemoff 7110 itemsize 41
        location key (507 INODE_ITEM 0)
        cache generation 445074 entries 169 bitmaps 3
    item 100 key (FREE_SPACE UNTYPED 36529242112) itemoff 7069 itemsize 41
        location key (538 INODE_ITEM 0)
        cache generation 444655 entries 36 bitmaps 0
    item 101 key (FREE_SPACE UNTYPED 46192918528) itemoff 7028 itemsize 41
        location key (547 INODE_ITEM 0)
        cache generation 445482 entries 14 bitmaps 0
    item 102 key (FREE_SPACE UNTYPED 47266660352) itemoff 6987 itemsize 41
        location key (548 INODE_ITEM 0)
        cache generation 444950 entries 25 bitmaps 0
    item 103 key (FREE_SPACE UNTYPED 48340402176) itemoff 6946 itemsize 41
        location key (550 INODE_ITEM 0)
        cache generation 444950 entries 33 bitmaps 0
    item 104 key (FREE_SPACE UNTYPED 49414144000) itemoff 6905 itemsize 41
        location key (620 INODE_ITEM 0)
        cache generation 445169 entries 67 bitmaps 0
    item 105 key (FREE_SPACE UNTYPED 50487885824) itemoff 6864 itemsize 41
        location key (622 INODE_ITEM 0)
        cache generation 445074 entries 118 bitmaps 1
    item 106 key (FREE_SPACE UNTYPED 51561627648) itemoff 6823 itemsize 41
        location key (625 INODE_ITEM 0)
        cache generation 445172 entries 144 bitmaps 3
    item 107 key (FREE_SPACE UNTYPED 52635369472) itemoff 6782 itemsize 41
        location key (629 INODE_ITEM 0)
        cache generation 445482 entries 81 bitmaps 1
    item 108 key (FREE_SPACE UNTYPED 53709111296) itemoff 6741 itemsize 41
        location key (637 INODE_ITEM 0)
        cache generation 445169 entries 46 bitmaps 0
    item 109 key (FREE_SPACE UNTYPED 54782853120) itemoff 6700 itemsize 41
        location key (641 INODE_ITEM 0)
        cache generation 445483 entries 6 bitmaps 0
    item 110 key (FREE_SPACE UNTYPED 55856594944) itemoff 6659 itemsize 41
        location key (643 INODE_ITEM 0)
        cache generation 444130 entries 8 bitmaps 0
    item 111 key (FREE_SPACE UNTYPED 56930336768) itemoff 6618 itemsize 41
        location key (666 INODE_ITEM 0)
        cache generation 444917 entries 2 bitmaps 0
    item 112 key (FREE_SPACE UNTYPED 58004078592) itemoff 6577 itemsize 41
        location key (680 INODE_ITEM 0)
        cache generation 445273 entries 4 bitmaps 0
    item 113 key (FREE_SPACE UNTYPED 59077820416) itemoff 6536 itemsize 41
        location key (682 INODE_ITEM 0)
        cache generation 445272 entries 5 bitmaps 0
    item 114 key (FREE_SPACE UNTYPED 60151562240) itemoff 6495 itemsize 41
        location key (687 INODE_ITEM 0)
        cache generation 444639 entries 1 bitmaps 0
    item 115 key (FREE_SPACE UNTYPED 61225304064) itemoff 6454 itemsize 41
        location key (688 INODE_ITEM 0)
        cache generation 444655 entries 70 bitmaps 0
    item 116 key (FREE_SPACE UNTYPED 62299045888) itemoff 6413 itemsize 41
        location key (704 INODE_ITEM 0)
        cache generation 434508 entries 0 bitmaps 0
    item 117 key (FREE_SPACE UNTYPED 63372787712) itemoff 6372 itemsize 41
        location key (705 INODE_ITEM 0)
        cache generation 434508 entries 0 bitmaps 0
    item 118 key (FREE_SPACE UNTYPED 64446529536) itemoff 6331 itemsize 41
        location key (706 INODE_ITEM 0)
        cache generation 444955 entries 2 bitmaps 0
    item 119 key (FREE_SPACE UNTYPED 65520271360) itemoff 6290 itemsize 41
        location key (707 INODE_ITEM 0)
        cache generation 444131 entries 4 bitmaps 0
    item 120 key (FREE_SPACE UNTYPED 66594013184) itemoff 6249 itemsize 41
        location key (708 INODE_ITEM 0)
        cache generation 445267 entries 78 bitmaps 0
    item 121 key (FREE_SPACE UNTYPED 67667755008) itemoff 6208 itemsize 41
        location key (709 INODE_ITEM 0)
        cache generation 444971 entries 30 bitmaps 0
    item 122 key (FREE_SPACE UNTYPED 68741496832) itemoff 6167 itemsize 41
        location key (710 INODE_ITEM 0)
        cache generation 444950 entries 43 bitmaps 0
    item 123 key (FREE_SPACE UNTYPED 69815238656) itemoff 6126 itemsize 41
        location key (712 INODE_ITEM 0)
        cache generation 444917 entries 109 bitmaps 1
    item 124 key (FREE_SPACE UNTYPED 70888980480) itemoff 6085 itemsize 41
        location key (715 INODE_ITEM 0)
        cache generation 444688 entries 10 bitmaps 0
    item 125 key (FREE_SPACE UNTYPED 71962722304) itemoff 6044 itemsize 41
        location key (716 INODE_ITEM 0)
        cache generation 444688 entries 16 bitmaps 0
    item 126 key (FREE_SPACE UNTYPED 73036464128) itemoff 6003 itemsize 41
        location key (717 INODE_ITEM 0)
        cache generation 437482 entries 0 bitmaps 0
    item 127 key (FREE_SPACE UNTYPED 74110205952) itemoff 5962 itemsize 41
        location key (718 INODE_ITEM 0)
        cache generation 437493 entries 0 bitmaps 0
    item 128 key (FREE_SPACE UNTYPED 75183947776) itemoff 5921 itemsize 41
        location key (719 INODE_ITEM 0)
        cache generation 437482 entries 0 bitmaps 0
    item 129 key (FREE_SPACE UNTYPED 76257689600) itemoff 5880 itemsize 41
        location key (720 INODE_ITEM 0)
        cache generation 444688 entries 51 bitmaps 0
    item 130 key (FREE_SPACE UNTYPED 77331431424) itemoff 5839 itemsize 41
        location key (721 INODE_ITEM 0)
        cache generation 444688 entries 98 bitmaps 0
    item 131 key (FREE_SPACE UNTYPED 78405173248) itemoff 5798 itemsize 41
        location key (722 INODE_ITEM 0)
        cache generation 445508 entries 86 bitmaps 0
    item 132 key (FREE_SPACE UNTYPED 79478915072) itemoff 5757 itemsize 41
        location key (723 INODE_ITEM 0)
        cache generation 445509 entries 13 bitmaps 0
    item 133 key (FREE_SPACE UNTYPED 80552656896) itemoff 5716 itemsize 41
        location key (724 INODE_ITEM 0)
        cache generation 444162 entries 25 bitmaps 0
    item 134 key (FREE_SPACE UNTYPED 81626398720) itemoff 5675 itemsize 41
        location key (725 INODE_ITEM 0)
        cache generation 444688 entries 5 bitmaps 0
    item 135 key (FREE_SPACE UNTYPED 83773882368) itemoff 5634 itemsize 41
        location key (727 INODE_ITEM 0)
        cache generation 444688 entries 7 bitmaps 0
    item 136 key (FREE_SPACE UNTYPED 84847624192) itemoff 5593 itemsize 41
        location key (728 INODE_ITEM 0)
        cache generation 444688 entries 5 bitmaps 0
    item 137 key (DATA_RELOC_TREE ROOT_ITEM 0) itemoff 5154 itemsize 439
        generation 4 root_dirid 256 bytenr 5390336 level 0 refs 1
        lastsnap 0 byte_limit 0 bytes_used 16384 flags 0x0(none)
        uuid 00000000-0000-0000-0000-000000000000
        drop key (0 UNKNOWN.0 0) level 0
mint@mint:~/Downloads/btrfsprogsgit/btrfs-progs$

adam900710 commented 3 years ago

OK, got the reason.

It looks like btrfs-progs can't handle half dropped trees, thus causing the problem.

The tree involved is

    item 94 key (617 ROOT_ITEM 355614) itemoff 3463 itemsize 439
        generation 421785 root_dirid 256 bytenr 12019957760 level 2 refs 0
        lastsnap 355614 byte_limit 0 bytes_used 454721536 flags 0x1000000000000(none)
        uuid c3256576-d4cf-2249-9a65-62df8bee6f59
        parent_uuid b4dc7e78-ea7e-dc43-8bd7-0580bb8e7b33
        ctransid 421785 otransid 355614 stransid 0 rtransid 0
        ctime 1614722401.977885503 (2021-03-02 23:00:01)
        otime 1594392677.269749085 (2020-07-10 16:51:17)
        drop key (20592873 INODE_REF 20592870) level 1

If your fs is still being able to be mounted RW, please mount it, run btrfs fi sync <mnt> which will wait for that tree to be deleted.

Then btrfs check should not trigger the qgroup problem.

SoulInfernoDE commented 3 years ago

Wow your reply is as fast as lightning. I'll reboot to prod system and will check there and reply here afterwards..

SoulInfernoDE commented 3 years ago

matze@laptop-matze:~$ btrfs version
btrfs-progs v5.11.1 
matze@laptop-matze:~$ sudo btrfs fi sync /dev/sda6
ERROR: Could not sync filesystem: Inappropriate ioctl for device
matze@laptop-matze:~$

adam900710 commented 3 years ago

You need to mount it, and execute the progs on the mount point, not the device.

SoulInfernoDE commented 3 years ago

Oh sorry my fault :+1:

I have done:

sudo btrfs fi sync / and its not complaining about io error.

So the output of btrfs check is:

matze@laptop-matze:~/Downloads/btrfs-progs/btrfs-progs$ sudo btrfs check --readonly --force /dev/sda6
Opening filesystem to check...
WARNING: filesystem mounted, continuing because of --force
Checking filesystem on /dev/sda6
UUID: 7677e01a-7637-43f3-b9aa-598e065e24ad
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
checksum verify failed on 12379783168 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 12379783168 wanted 0x00000000 found 0xb6bde3e4
bad tree block 12379783168, bytenr mismatch, want=12379783168, have=0
check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1
btrfs(+0x1bb6e)[0x55657e3edb6e]
btrfs(qgroup_verify_all+0x4c6)[0x55657e3ef343]
btrfs(+0x4b2e7)[0x55657e41d2e7]
btrfs(main+0x98)[0x55657e3e9f08]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fe03af990b3]
btrfs(_start+0x2e)[0x55657e3e9a8e]
Abgebrochen
matze@laptop-matze:~/Downloads/btrfs-progs/btrfs-progs$

It did indeed help. Only two lines left to fix instead of hundreds. Maybe you have an idea for that also? :+1: Could it be that i forced a --readonly check on prod. system? ..im not sure if i should reboot and check if the message persists in the live env?

best regards SI

adam900710 commented 3 years ago

It could be related to the --readonly, normally we don't recommand using that option, unless it's unavoidable.

Thus still recommended to do a proper btrfs check on the unmounted fs.

Anyway, this should not be a big thing, as qgroup shouldn't really cause any obvious impact.

SoulInfernoDE commented 3 years ago

Okay i rebooted and tried to btrfs check again on unmounted /dev/sda6

mint@mint:~/Downloads$ sudo ./btrfs check /dev/sda6
Opening filesystem to check...
Checking filesystem on /dev/sda6
UUID: 7677e01a-7637-43f3-b9aa-598e065e24ad
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
checksum verify failed on 12379783168 wanted 0x00000000 found 0xb6bde3e4
checksum verify failed on 12379783168 wanted 0x00000000 found 0xb6bde3e4
bad tree block 12379783168, bytenr mismatch, want=12379783168, have=0
check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1
./btrfs(+0x1bf22)[0x55a901653f22]
./btrfs(qgroup_verify_all+0x4c6)[0x55a9016556f7]
./btrfs(+0x4b69b)[0x55a90168369b]
./btrfs(main+0x3c9)[0x55a901650555]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f3b676e70b3]
./btrfs(_start+0x2e)[0x55a90164fd6e]
Abgebrochen

I am not sure if Clonezilla will complete when this errors are seen, because at the end of the command it says: Abgebrochen which means in english something like: cancelled

adam900710 commented 3 years ago

For the worst case, you can try to disable qgroup by btrfs quota disable <mnt> which should remove qgroup completely.

Then you can check if the tool still works.

As despite the qgroup tree block read error, there seems to be nothing wrong at all of your fs.

SoulInfernoDE commented 3 years ago

Will do that tomorrow as i need to do some rl stuff now. I'll report back if it worked. :+1: Despite that do you think it is also kind of bug related message or should btrfs-progs behave like that and this message is normal and should be printed in any and future situation/releases of btrfs-progs commands ?

adam900710 commented 3 years ago

It should be a btrfs-progs bug, firstly it didn't follow kernel dropped trees (this is kinda known). Secondly it should not cause BUG_ON() for when hitting read errors.

Will look into the problem soon.

SoulInfernoDE commented 3 years ago

Hi @adam900710, so i tried again to backup the partition and it throws error and cancelled the clone process. I then disabled qgroup and the same error happened. Clonezilla is throwing errors and aborts. It also makes the clone process until it aborts extremely slow (very slow!). 125gb needed eight hours and then aborts after thousands of errors.

I did a qgroup rescan /dev/sda6 which cleared the two errors mentioned above. I then tried to btrfs check /dev/sda6

mint@mint:~$ sudo btrfs check /dev/sda6
Opening filesystem to check...
Checking filesystem on /dev/sda6
UUID: 7677e01a-7637-43f3-b9aa-598e065e24ad
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1
btrfs(+0x18fb9)[0x5566f5cbffb9]
btrfs(+0x18fea)[0x5566f5cbffea]
btrfs(+0x19177)[0x5566f5cc0177]
btrfs(qgroup_verify_all+0x486)[0x5566f5cc1486]
btrfs(+0x3e0fb)[0x5566f5ce50fb]
btrfs(main+0x2a2)[0x5566f5cbcc62]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f7a0a50c0b3]
btrfs(_start+0x2e)[0x5566f5cbcc9e]
Abgebrochen

which also cancelled the scan. But now i don't see any plausible error now.. Hmh ..will update my post as soon as i tried now to backup with qgroup disabled again after rescan. Also is it save to destroy my qgroups before disable in case to fix the abort?

mint@mint:~$ sudo btrfs qgroup show /mnt/sda6
qgroupid         rfer         excl 
--------         ----         ---- 
0/5          32.00KiB     32.00KiB 
0/257        58.14GiB    321.83MiB 
0/671        16.00KiB     16.00KiB 
0/672        16.00KiB     16.00KiB 
0/729        41.23GiB    198.99MiB 
0/730        41.33GiB    173.90MiB 
0/731        41.20GiB    256.62MiB 
0/742        58.07GiB    211.50MiB

btrfs check /dev/sda6 with qgroup disabled went through the check at least

mint@mint:~$ sudo btrfs check /dev/sda6
Opening filesystem to check...
Checking filesystem on /dev/sda6
UUID: 7677e01a-7637-43f3-b9aa-598e065e24ad
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 84640731136 bytes used, no error found
total csum bytes: 74560608
total tree bytes: 1673887744
total fs tree bytes: 1514438656
total extent tree bytes: 69074944
btree space waste bytes: 386667662
file data blocks allocated: 156428750848
 referenced 132420378624

adam900710 commented 3 years ago

So there are two bugs, and even maybe unrelated.

For the clonezilla bug, I have no idea as I never use it.

For the qgroup part, if your fs contains no confidential filenames, would you mind to provide the btrfs-image dump of it?

# btrfs-image /dev/sda6 <output_file>

Please note that, although this dump won't provide any data, but it contains file and directory names.

ShrirajHegde commented 2 years ago

For the worst case, you can try to disable qgroup by btrfs quota disable <mnt> which should remove qgroup completely.

Then you can check if the tool still works.

As despite the qgroup tree block read error, there seems to be nothing wrong at all of your fs.

I have the exact same problem.

[5/7] checking csums (without verifying data)  (0:00:00 elapsed, 1077 items checked)
[5/7] checking csums (without verifying data)  (0:00:00 elapsed, 393376 items checked)
[6/7] checking root refs                       (0:00:00 elapsed, 10 items checked)
[6/7] checking root refs                       (0:00:00 elapsed, 10 items checked)
[7/7] checking quota groups                    (0:00:00 elapsed)
[7/7] checking quota groups                    (0:00:01 elapsed, 146609 items checked)
[7/7] checking quota groups                    (0:00:02 elapsed, 279894 items checked)
check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1
btrfs(+0x5387a)[0x55dbc190b87a]
btrfs(qgroup_verify_all+0x614)[0x55dbc190c3f4]
btrfs(+0x840bd)[0x55dbc193c0bd]
btrfs(main+0x8d)[0x55dbc18cfc9d]
/lib64/libc.so.6(+0x2d560)[0x7feee1229560]
/lib64/libc.so.6(__libc_start_main+0x7c)[0x7feee122960c]
btrfs(_start+0x25)[0x55dbc18cfff5]

And, this error does occur with quota disabled. Disabling and then re-enabling quota brings back the issue. I have tried upto btrfs-progs v5.15.1

Is there any other fix that disabling quota? Because I use timeshift, and it works better with quota enabled.

kdave / btrfs-progs

check/qgroup-verify.c:546: account_all_refs: BUG_ON `ref->num_bytes != num_bytes` triggered, value 1 #358