search

openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.65k stars 1.75k forks source link

Unable to expand alloc size and iostat display error #13468

Closed Seaward233 closed 2 years ago

Seaward233 commented 2 years ago

System information

Type Version/Name
Distribution Name Debian
Distribution Version 4.19.208-1 (2021-09-29)
Kernel Version 4.19.0-18-amd64
Architecture x86
OpenZFS Version 2.0.3-9~bpo10+1

Describe the problem you're observing

1.Unable to expand after disk swap.

Describe how to reproduce the problem

With autoexpend=on, offline sdh, shutdown for replacement, turn back on after replace, run zpool replace sdh, resilver without error but not expend. sdd sdf sdh are 4TB drives.

I tried zpool online -e on all drive and reboot, nothing works. I also tried to remove Solaris reserved 1 partition, nothing works.

Include any warning/errors/backtraces from the system logs

ansme@mediavault ~> sudo zpool list -v
NAME                                      SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
tank                                     32.6T  20.8T  11.9T        -         -     0%    63%  1.00x    ONLINE  -
  raidz1                                 21.8T  15.4T  6.35T        -         -     0%  70.8%      -  ONLINE
    sdc                                      -      -      -        -         -      -      -      -  ONLINE
    sde                                      -      -      -        -         -      -      -      -  ONLINE
    sdg                                      -      -      -        -         -      -      -      -  ONLINE
  raidz1                                 10.9T  5.35T  5.52T        -         -     0%  49.2%      -  ONLINE
    sdd                                      -      -      -        -         -      -      -      -  ONLINE
    sdf                                      -      -      -        -         -      -      -      -  ONLINE
    sdh                                      -      -      -        -         -      -      -      -  ONLINE
logs                                         -      -      -        -         -      -      -      -  -
  mirror                                 3.97G    36K  3.97G        -         -     0%  0.00%      -  ONLINE
    ata-Q200_EX._37NS133JT1ET-part3          -      -      -        -         -      -      -      -  ONLINE
    ata-TOSHIBA_Q300_65QB5105KNSX-part3      -      -      -        -         -      -      -      -  ONLINE
cache                                        -      -      -        -         -      -      -      -  -
  ata-Q200_EX._37NS133JT1ET-part4         189G   189G  91.3M        -         -     0%  100.0%      -  ONLINE
  ata-TOSHIBA_Q300_65QB5105KNSX-part4     211G   211G  95.7M        -         -     0%  100.0%      -  ONLINE
ansme@mediavault ~> sudo fdisk -l
Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Disk model: TOSHIBA Q300
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: B4821FCC-40B1-45EB-80FC-D139DC397D2E

Device        Start       End   Sectors   Size Type
/dev/sda1      2048   1050623   1048576   512M Linux filesystem
/dev/sda2   1050624  17827839  16777216     8G Linux swap
/dev/sda3  17827840  26216447   8388608     4G Linux filesystem
/dev/sda4  26216448 468860927 442644480 211.1G Linux filesystem

Disk /dev/sdc: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: WDC WD80EZAZ-11T
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 32FB1CB2-03F8-7247-A34A-6F3AA375D720

Device     Start         End     Sectors  Size Type
/dev/sdc1   2048 15628036095 15628034048  7.3T Solaris /usr & Apple ZFS

Disk /dev/sdb: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Disk model: Q200 EX.
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 93DC1D79-CB35-4BAB-94E9-0A6E0370DFE2

Device        Start       End   Sectors   Size Type
/dev/sdb1      2048   1050623   1048576   512M EFI System
/dev/sdb2   1050624  63965183  62914560    30G Linux filesystem
/dev/sdb3  63965184  72353791   8388608     4G Linux filesystem
/dev/sdb4  72353792 468860927 396507136 189.1G Linux filesystem

Disk /dev/sdd: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: WDC WD40EFRX-68N
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 8C81B443-9051-E442-B10C-53082C492403

Device     Start        End    Sectors  Size Type
/dev/sdd1   2048 7814019071 7814017024  3.7T Solaris /usr & Apple ZFS

Disk /dev/sde: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: WDC WD80EZAZ-11T
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 3EF297C1-A4B4-6340-8C3D-55F6DF8E7119

Device     Start         End     Sectors  Size Type
/dev/sde1   2048 15628036095 15628034048  7.3T Solaris /usr & Apple ZFS

Disk /dev/sdf: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: HGST HDN724040AL
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: B3E2F0EC-7214-B843-886B-C6F5297A40C3

Device     Start        End    Sectors  Size Type
/dev/sdf1   2048 7814019071 7814017024  3.7T Solaris /usr & Apple ZFS

Disk /dev/sdg: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: WDC WD80EZAZ-11T
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 91C0DA23-859D-3D47-A94B-83D7CB6CB4EA

Device     Start         End     Sectors  Size Type
/dev/sdg1   2048 15628036095 15628034048  7.3T Solaris /usr & Apple ZFS

Disk /dev/sdh: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000NM0053
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 51BAD06A-637B-B842-AF8D-B5A6FEB74709

Device     Start        End    Sectors  Size Type
/dev/sdh1   2048 7814019071 7814017024  3.7T Solaris /usr & Apple ZFS
rincebrain commented 2 years ago

It's not clear to me what you think is wrong here.

The second raidz1 vdev is reporting "10.9T " total, which is about 3.7T 3 - in particular, if you ask Google for that number of bytes 3 in tebibytes, it says "10.9".

iostat/list alignment being buggy sometimes is not surprising to me though, having written a filter script to make it line up better (among other things). It's probably not too bad a fix, but not the biggest issue at hand.

Seaward233 commented 2 years ago

It's not clear to me what you think is wrong here.

The second raidz1 vdev is reporting "10.9T " total, which is about 3.7T 3 - in particular, if you ask Google for that number of bytes 3 in tebibytes, it says "10.9".

iostat/list alignment being buggy sometimes is not surprising to me though, having written a filter script to make it line up better (among other things). It's probably not too bad a fix, but not the biggest issue at hand.

Thanks for your reply, looks like I do the math wrong😂.