kdave / btrfs-progs

Development of userspace BTRFS tools
GNU General Public License v2.0
557 stars 242 forks source link

btrfs check --check-data-csum on raid 5/6 #346

Open drivezt opened 3 years ago

drivezt commented 3 years ago

On raid 5/6 command btrfs check --check-data-csum show error:

mirror 2 bytenr NNNNNNN csum NN expected csum NN
ERROR: errors found in csum tree

for example:

# mkfs.btrfs --metadata raid10 --data raid5 /dev/sd[bcde] -f;\
mount /dev/sdb /mnt; dd if=/dev/urandom of=/mnt/zfile bs=1K count=70;\
umount /mnt; btrfs check --check-data-csum /dev/sdb;\
mount /dev/sdb /mnt; btrfs device stats /mnt; umount /mnt
btrfs-progs v5.10.1
See http://btrfs.wiki.kernel.org for more information.

Label:              (null)
UUID:               7a01ce0e-63b8-4911-abcd-27eff8b921c3
Node size:          16384
Sector size:        4096
Filesystem size:    32.00GiB
Block group profiles:
  Data:             RAID5             3.00GiB
  Metadata:         RAID10          128.00MiB
  System:           RAID10           16.00MiB
SSD detected:       no
Incompat features:  extref, raid56, skinny-metadata
Runtime features:
Checksum:           crc32c
Number of devices:  4
Devices:
   ID        SIZE  PATH
    1     8.00GiB  /dev/sdb
    2     8.00GiB  /dev/sdc
    3     8.00GiB  /dev/sdd
    4     8.00GiB  /dev/sde

70+0 записей получено
70+0 записей отправлено
71680 байт (72 kB, 70 KiB) скопирован, 0,00690105 s, 10,4 MB/s
Opening filesystem to check...
Checking filesystem on /dev/sdb
UUID: 7a01ce0e-63b8-4911-abcd-27eff8b921c3
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking csums against data
mirror 2 bytenr 173015040 csum 42 expected csum 18
mirror 2 bytenr 173019136 csum 247 expected csum 60
mirror 2 bytenr 173080576 csum 42 expected csum 177
mirror 2 bytenr 173084672 csum 247 expected csum 66
ERROR: errors found in csum tree
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 1056768 bytes used, error(s) found
total csum bytes: 72
total tree bytes: 131072
total fs tree bytes: 32768
total extent tree bytes: 16384
btree space waste bytes: 122423
file data blocks allocated: 925696
 referenced 925696
[/dev/sdb].write_io_errs    0
[/dev/sdb].read_io_errs     0
[/dev/sdb].flush_io_errs    0
[/dev/sdb].corruption_errs  0
[/dev/sdb].generation_errs  0
[/dev/sdc].write_io_errs    0
[/dev/sdc].read_io_errs     0
[/dev/sdc].flush_io_errs    0
[/dev/sdc].corruption_errs  0
[/dev/sdc].generation_errs  0
[/dev/sdd].write_io_errs    0
[/dev/sdd].read_io_errs     0
[/dev/sdd].flush_io_errs    0
[/dev/sdd].corruption_errs  0
[/dev/sdd].generation_errs  0
[/dev/sde].write_io_errs    0
[/dev/sde].read_io_errs     0
[/dev/sde].flush_io_errs    0
[/dev/sde].corruption_errs  0
[/dev/sde].generation_errs  0

more data -> more errors (count > 60) data profiles raid 0,1,10,single,1c3,1c4 don't have errors with "check-data-csum" metadata profile does not matter

Checked on Linux 5.10, 5.11 (Arch Linux) and openSUSE Tumbleweed

kaklik commented 3 years ago

The same happens to me:

# btrfs --version
btrfs-progs v5.7 

This command gives me the errors. The output is shortened:

root@turris:~# btrfs check --check-data-csum /dev/sde
mirror 3 bytenr 1055862259712 csum 235 expected csum 101
mirror 3 bytenr 1055862263808 csum 149 expected csum 107
mirror 3 bytenr 1055862267904 csum 155 expected csum 133
mirror 3 bytenr 1055862272000 csum 81 expected csum 136
mirror 3 bytenr 1055862276096 csum 193 expected csum 43
mirror 3 bytenr 1055862280192 csum 192 expected csum 245
mirror 3 bytenr 1055862284288 csum 69 expected csum 88
mirror 3 bytenr 1055862288384 csum 220 expected csum 233

The parameters of the filesystem:

root@turris:~# btrfs filesystem usage /data
Overall:
    Device size:                  13.64TiB
    Device allocated:              1.80TiB
    Device unallocated:           11.84TiB
    Device missing:                  0.00B
    Used:                          1.77TiB
    Free (estimated):              7.12TiB      (min: 5.94TiB)
    Data ratio:                       1.67
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)
    Multiple profiles:                  no

Data,RAID6: Size:1.08TiB, Used:1.06TiB (98.62%)
   /dev/sdb      368.00GiB
   /dev/sdc      368.00GiB
   /dev/sdd      368.00GiB
   /dev/sde      368.00GiB
   /dev/sdf      368.00GiB

Metadata,RAID1: Size:3.00GiB, Used:1.62GiB (53.96%)
   /dev/sdb        2.00GiB
   /dev/sdc        1.00GiB
   /dev/sdd        1.00GiB
   /dev/sde        1.00GiB
   /dev/sdf        1.00GiB

System,RAID1: Size:32.00MiB, Used:112.00KiB (0.34%)
   /dev/sdc       32.00MiB
   /dev/sdd       32.00MiB

Unallocated:
   /dev/sdb        2.37TiB
   /dev/sdc        2.37TiB
   /dev/sdd        2.37TiB
   /dev/sde        2.37TiB
   /dev/sdf        2.37TiB
root@turris:~#