search

openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.52k stars 1.74k forks source link

expandsize=16.0E #1468

Closed FransUrbo closed 5 years ago

FransUrbo commented 11 years ago

I have (and had) autoexpand=on, but still get expandsize=16.0E. I replaced three 1.5TB disks with tree 3TB disks about a month ago, and i did get extra space (I'm not 100% sure I got the size I had expected/should but...).

How much is this actually, and how do I know where this available space is?

I found Illumos #1948 (ZoL pull #908), but that's closed and don't seem to have been merged into ZoL. In that, it seems that '16E' just means undefined.. Do I actually have free/unallocated space or not?

FransUrbo commented 11 years ago

Please see thread on ZoL list with the same subject for possible reason.

Gregor Kopka:
The expandsize <> 0 you see looks like an artifact (looks like small negative signed integer - maybe from mixup of partition and disk sizes - interpreted as positive MAXINT) .
behlendorf commented 11 years ago

Where does this stand? Is this still an issue or was it explained?

FransUrbo commented 11 years ago

It's still an issue.

celia:~# zpool get expandsize share
NAME   PROPERTY    VALUE     SOURCE
share  expandsize  16.0E     -
behlendorf commented 11 years ago

Have you manually run zpool online -e poolname vdev for the replaced vdevs? The autoexpand feature in ZoL currently doesn't yet do this automatically for you.

FransUrbo commented 11 years ago

I have now, and it's still 16.0E.

behlendorf commented 11 years ago

Somewhere it's definitely being miscalculated, we'll need to run it down.

FransUrbo commented 10 years ago

I can't reproduce this on my test rig with latest HEAD.

didyouexpectthat commented 8 years ago

I ran into this today. I had 2 SAS disks in mirror, then I removed one of the disks so I could do some other things in that SAS channel... added the disk back to the rpool and finished resilvering, now I notice this.

root@goliath:~# zpool list rpool
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool   832G   336G   496G     16.0E    51%    40%  1.00x  ONLINE  -
root@goliath:~# zpool get all rpool | grep expand -i
rpool  autoexpand                  off                         default
rpool  expandsize                  16.0E                       -
root@goliath:~# dpkg -l | grep zfs
ii  libzfs2                        0.6.5-pve7~jessie              amd64        Native ZFS filesystem library for Linux
ii  zfs-initramfs                  0.6.5-pve7~jessie              amd64        Native ZFS root filesystem capabilities for Linux
ii  zfsutils                       0.6.5-pve7~jessie              amd64        command-line tools to manage ZFS filesystems
flixman commented 7 years ago

I have exactly the same problem: I am running zol-0.6.5.8 and, after having migrated a file-based zpool to a mirror zpool (files were 60GB, drives are 64GB) I have set autoexpand to on for the pool and run the zfs pool online -e for both disks. Can this issue be reopened?

behlendorf commented 7 years ago

Reopening. It would be helpful if someone could come up with a simple test case which reproduces this.

didyouexpectthat commented 7 years ago

As a follow up, mine no longer shows this way. I use Proxmox VE 4.3 and periodically dist-upgrade. I wouldn't exactly know which version it displayed correctly at. 

root@goliath:/# zpool list rpool; zpool get all rpool | grep expand -i; dpkg -l | grep zfs
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool   832G   322G   510G         -    55%    38%  1.00x  ONLINE  -
rpool  autoexpand                  off                         default
rpool  expandsize                  -                           -
ii  libzfs2                              0.6.5.8-pve13~bpo80            all          transitional package
ii  libzfs2linux                         0.6.5.8-pve13~bpo80            amd64        OpenZFS filesystem library for Linux
ii  zfs-initramfs                        0.6.5.8-pve13~bpo80            all          OpenZFS root filesystem capabilities for Linux - initramfs
ii  zfs-zed                              0.6.5.8-pve13~bpo80            amd64        OpenZFS Event Daemon
ii  zfsutils                             0.6.5.8-pve13~bpo80            all          transitional package
ii  zfsutils-linux                       0.6.5.8-pve13~bpo80            amd64        command-line tools to manage OpenZFS filesystems
dorvan commented 7 years ago

Some data on same problem, same steps and procedure, on different pools and disks. normal operations after an upgrade: 

zpool upgrade -a
zfs upgrade -a

Commands and Data:


NAME        SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
Storage01  1.81T  1.48T   342G     16.0E    73%    81%  2.05x  ONLINE  /Data/Storage01
Storage02   928G   185G   743G         -    14%    19%  1.28x  ONLINE  /Data/Storage02
Storage03  1.77T  1.08T   698G         -    48%    61%  1.90x  ONLINE  /Data/Storage03

:pts/0: [root@asrvca03] ~ #> zpool status
  pool: Storage01
 state: ONLINE
  scan: scrub repaired 0 in 25h40m with 0 errors on Fri Jun 10 11:32:02 2016
config:

        NAME        STATE     READ WRITE CKSUM
        Storage01   ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sdc1    ONLINE       0     0     0
            sdd1    ONLINE       0     0     0

errors: No known data errors

  pool: Storage02
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        Storage02   ONLINE       0     0     0
          sda1      ONLINE       0     0     0

errors: No known data errors

  pool: Storage03
 state: ONLINE
  scan: scrub repaired 0 in 9h38m with 0 errors on Fri Jul  8 19:39:05 2016
config:

        NAME        STATE     READ WRITE CKSUM
        Storage03   ONLINE       0     0     0
          sdb3      ONLINE       0     0     0

errors: No known data errors

Welcome to fdisk (util-linux 2.28).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 02D696B3-4415-4B57-A74E-CC0BE25DB91A

Device     Start        End    Sectors  Size Type
/dev/sdc1   2048 3907029134 3907027087  1.8T Linux filesystem

Welcome to fdisk (util-linux 2.28).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sdd: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: DC61E4C0-7B6F-4C8A-8506-59723EBA799D

Device     Start        End    Sectors  Size Type
/dev/sdd1   2048 3907026943 3907024896  1.8T Linux filesystem

Welcome to fdisk (util-linux 2.28).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x000c011d

Device     Boot    Start        End    Sectors  Size Id Type
/dev/sdb3       93749248 3907028991 3813279744  1.8T 83 Linux <-- without problem

[   16.458272] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)
[   16.458296] sd 0:0:0:0: Attached scsi generic sg0 type 0
[   16.458311] sd 0:0:0:0: [sda] Write Protect is off
[   16.458316] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   16.458335] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   16.513529]  sda: sda1
[   16.513932] sd 0:0:0:0: [sda] Attached SCSI disk
[   16.601406] sd 1:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[   16.601413] sd 1:0:0:0: [sdb] 4096-byte physical blocks
[   16.601425] sd 1:0:0:0: Attached scsi generic sg1 type 0
[   16.601451] sd 1:0:0:0: [sdb] Write Protect is off
[   16.601456] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[   16.601472] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   16.602767] sd 2:0:0:0: [sdc] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[   16.602774] sd 2:0:0:0: [sdc] 4096-byte physical blocks
[   16.602803] sd 2:0:0:0: [sdc] Write Protect is off
[   16.602807] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[   16.602820] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   16.602996] sd 2:0:0:0: Attached scsi generic sg2 type 0
[   16.612960]  sdb: sdb3
[   16.613311] sd 1:0:0:0: [sdb] Attached SCSI disk
[   16.641188] sd 3:0:0:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[   16.641194] sd 3:0:0:0: [sdd] 4096-byte physical blocks
[   16.641219] sd 3:0:0:0: [sdd] Write Protect is off
[   16.641222] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[   16.641234] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   16.641488] sd 3:0:0:0: Attached scsi generic sg3 type 0
[   16.657595]  sdc: sdc1
[   16.658038] sd 2:0:0:0: [sdc] Attached SCSI disk
[   16.697479]  sdd: sdd1
[   16.697901] sd 3:0:0:0: [sdd] Attached SCSI disk

:pts/0: [root@asrvca03] ~ #> hdparm -iI /dev/sdc

/dev/sdc:

 Model=ST2000DM001-1CH164, FwRev=CC47, SerialNo=Z240WE6F
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=3907029168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: Reserved:  ATA/ATAPI-4,5,6,7

 * signifies the current active mode

ATA device, with non-removable media
        Model Number:       ST2000DM001-1CH164
        Serial Number:      Z240WE6F
        Firmware Revision:  CC47
        Transport:          Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
        Used: unknown (minor revision code 0x001f)
        Supported: 9 8 7 6 5
        Likely used: 9
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  268435455
        LBA48  user addressable sectors: 3907029168
        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:     1907729 MBytes
        device size with M = 1000*1000:     2000398 MBytes (2000 GB)
        cache/buffer size  = unknown
        Form Factor: 3.5 inch
        Nominal Media Rotation Rate: 7200
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, no device specific minimum
        R/W multiple sector transfer: Max = 16  Current = ?
        Advanced power management level: 128
        Recommended acoustic management value: 208, current value: 0
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    DOWNLOAD_MICROCODE
           *    Advanced Power Management feature set
                Power-Up In Standby feature set
           *    SET_FEATURES required to spinup after power up
                SET_MAX security extension
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    General Purpose Logging feature set
           *    WRITE_{DMA|MULTIPLE}_FUA_EXT
           *    64-bit World wide name
                Write-Read-Verify feature set
           *    WRITE_UNCORRECTABLE_EXT command
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Phy event counters
           *    READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
           *    DMA Setup Auto-Activate optimization
                Device-initiated interface power management
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Write Same (AC2)
                unknown 206[7]
                unknown 206[12] (vendor specific)
                unknown 206[13] (vendor specific)
Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
                supported: enhanced erase
        244min for SECURITY ERASE UNIT. 244min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000c5006458bc65
        NAA             : 5
        IEEE OUI        : 000c50
        Unique ID       : 06458bc65
Checksum: correct
:pts/0: [root@asrvca03] ~ #> hdparm -iI /dev/sdd

/dev/sdd:

 Model=ST2000DM001-9YN164, FwRev=CC4G, SerialNo=Z1E1LV41
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=3907029168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-4,5,6,7

 * signifies the current active mode

ATA device, with non-removable media
        Model Number:       ST2000DM001-9YN164
        Serial Number:      Z1E1LV41
        Firmware Revision:  CC4G
        Transport:          Serial, SATA Rev 3.0
Standards:
        Used: unknown (minor revision code 0x0029)
        Supported: 8 7 6 5
        Likely used: 8
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  268435455
        LBA48  user addressable sectors: 3907029168
        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:     1907729 MBytes
        device size with M = 1000*1000:     2000398 MBytes (2000 GB)
        cache/buffer size  = unknown
        Nominal Media Rotation Rate: 7200
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, no device specific minimum
        R/W multiple sector transfer: Max = 16  Current = ?
        Advanced power management level: 128
        Recommended acoustic management value: 208, current value: 0
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    DOWNLOAD_MICROCODE
           *    Advanced Power Management feature set
                SET_MAX security extension
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    General Purpose Logging feature set
           *    WRITE_{DMA|MULTIPLE}_FUA_EXT
           *    64-bit World wide name
                Write-Read-Verify feature set
           *    WRITE_UNCORRECTABLE_EXT command
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Phy event counters
           *    READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
           *    DMA Setup Auto-Activate optimization
                Device-initiated interface power management
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Write Same (AC2)
                unknown 206[7]
                unknown 206[12] (vendor specific)
                unknown 206[13] (vendor specific)
Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
                supported: enhanced erase
        222min for SECURITY ERASE UNIT. 222min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000c5004e89008e
        NAA             : 5
        IEEE OUI        : 000c50
        Unique ID       : 04e89008e
Checksum: correct
:pts/0: [root@asrvca03] ~ #> hdparm -iI /dev/sdb

/dev/sdb:

 Model=ST2000DM001-1CH164, FwRev=CC24, SerialNo=S1E1G5EH
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=3907029168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-4,5,6,7

 * signifies the current active mode

ATA device, with non-removable media
        Model Number:       ST2000DM001-1CH164
        Serial Number:      S1E1G5EH
        Firmware Revision:  CC24
        Transport:          Serial, SATA Rev 3.0
Standards:
        Used: unknown (minor revision code 0x0029)
        Supported: 8 7 6 5
        Likely used: 8
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  268435455
        LBA48  user addressable sectors: 3907029168
        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:     1907729 MBytes
        device size with M = 1000*1000:     2000398 MBytes (2000 GB)
        cache/buffer size  = unknown
        Form Factor: 3.5 inch
        Nominal Media Rotation Rate: 7200
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, no device specific minimum
        R/W multiple sector transfer: Max = 16  Current = ?
        Advanced power management level: 128
        Recommended acoustic management value: 208, current value: 0
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    DOWNLOAD_MICROCODE
           *    Advanced Power Management feature set
                SET_MAX security extension
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    General Purpose Logging feature set
           *    WRITE_{DMA|MULTIPLE}_FUA_EXT
           *    64-bit World wide name
                Write-Read-Verify feature set
           *    WRITE_UNCORRECTABLE_EXT command
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Phy event counters
           *    READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
           *    DMA Setup Auto-Activate optimization
                Device-initiated interface power management
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Write Same (AC2)
                unknown 206[7]
                unknown 206[12] (vendor specific)
                unknown 206[13] (vendor specific)
Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
                supported: enhanced erase
        222min for SECURITY ERASE UNIT. 222min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000c5005c1b9b6a
        NAA             : 5
        IEEE OUI        : 000c50
        Unique ID       : 05c1b9b6a
Checksum: correct```
loli10K commented 7 years ago

This may be fixed in https://github.com/zfsonlinux/zfs/commit/2e215fecbe97167cd5408cea19aa50b80601248c.

behlendorf commented 6 years ago

Closing, this should be fixed.

DeHackEd commented 5 years ago

I am experiencing this on the current Git (0.8.0-rc2_22_gd649604).

# zpool create -f -O compress=lz4 -O mountpoint=/srv -O atime=off -O canmount=off -O sync=disabled -O redundant_metadata=most -O xattr=sa -O recordsize=4M \
  -d -o feature@async_destroy=enabled -o feature@lz4_compress=enabled -o feature@empty_bpobj=enabled -o feature@spacemap_histogram=enabled -o feature@bookmarks=enabled -o feature@embedded_data=enabled -o feature@large_blocks=enabled -o feature@allocation_classes=enabled $POOLNAME \
  raidz2 [10 disks] \
  raidz2 [10 disks] \
  [12 instances of raidz2 total] \
  spare [4 disks]
# zpool add $POOLNAME -o ashift=9 special mirror [2 disks]

# zpool list -vp
NAME                     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
poolname                599478650273792  581370672128  598897279601664        -  18446740775174668288      0      0   1.00    ONLINE  -
  raidz2                39857296506880  40340140032  39816956366848        -  18446743798831644672      0      0      -  ONLINE  
    disk1                   -      -      -        -  18446743798831644672      -      -      -  ONLINE  
    [....]
[...]
special                     -      -      -        -         -      -      -      -  -
  mirror                794568949760  508470272  794060479488        -         -      0      0      -  ONLINE  
    disk121                 -      -      -        -         -      -      -      -  ONLINE  
    disk122                 -      -      -        -  18446744069414584320      -      -      -  ONLINE  
spare                       -      -      -        -         -      -      -      -  -
  disk123                   -      -      -        -         -      -      -      -  AVAIL   
  disk124                   -      -      -        -         -      -      -      -  AVAIL   
  disk125                   -      -      -        -         -      -      -      -  AVAIL   
  disk126                   -      -      -        -         -      -      -      -  AVAIL

It's worth nothing there's actually a lot of disk size variation. I have some 4TB disks and some 8TB disks. 9 raidz2 arrays are made of 4TB disks, and 3 are made of 8TB disks. One of the "4TB raidz2" arrays is made of 9x 4TB disks and 1x 8TB disk because... well that happened. Also the Special mirror disks have different sizes.