search

morey-tech / homelab

0 stars 0 forks source link

Expand `storage-mass` zpool with 10TB disks #34

Closed morey-tech closed 4 months ago

morey-tech commented 4 months ago

The existing storage-mass zpool uses 4TB drives.

root@helios:~# zpool status
  pool: storage-mass
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 06:11:18 with 0 errors on Sun May 12 06:35:20 2024
config:

        NAME                        STATE     READ WRITE CKSUM
        storage-mass                ONLINE       0     0     0
          raidz2-0                  ONLINE       0     0     0
            wwn-0x5000c5007a41d223  ONLINE       0     0     0
            wwn-0x5000c5007a4232c4  ONLINE       0     0     0
            wwn-0x5000c5007a424ed9  ONLINE       0     0     0
            wwn-0x5000c5007a476cc0  ONLINE       0     0     0

Grow the zpool by replacing the existing drives with the ones currently in use by storage-mass-new zpool.

  pool: storage-mass-new
 state: ONLINE
config:

        NAME                                    STATE     READ WRITE CKSUM
        storage-mass-new                        ONLINE       0     0     0
          raidz1-0                              ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCG9YGZM  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGAM7KM  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGSDUGN  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCH2NUVP  ONLINE       0     0     0

Replace disks with larger ones This is what you're discussing in your question. It's the normal way of growing a ZFS pool when you have a pool layout that you are happy with.

To replace a device with a new one, the new device needs to be at least as large as the old one.

Operationally, you'd hook up the new disk along with the old, and then zpool replace the old disk with the new one. (This creates a temporary replacing device which becomes a parent to the old and new disk; when the resilver completes, the replacing device is removed from the device tree and it looks like the new device was there all along.) Once the resilver completes, the old disk can be removed from the system.

Once all disks in a vdev are replaced by larger ones, you can expand the pool by running zpool online -e or by having the autoexpand property set to on (though I wouldn't really recommend the latter; pool expansion should be a conscious decision). https://unix.stackexchange.com/a/532601

morey-tech commented 4 months ago

Destroyed the temporary pool:

root@helios:~# zpool destroy storage-mass-new

Example disk replacement:

root@helios:~# zpool replace storage-mass wwn-0x5000c5007a41d223 ata-WDC_WD101EMAZ-11G7DA0_VCG9YGZM
root@helios:~# zpool status
  pool: storage-mass
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sun May 26 13:26:56 2024
        1.85T / 10.1T scanned at 99.8G/s, 0B / 10.1T issued
        0B resilvered, 0.00% done, no estimated completion time
config:

        NAME                                      STATE     READ WRITE CKSUM
        storage-mass                              ONLINE       0     0     0
          raidz2-0                                ONLINE       0     0     0
            replacing-0                           ONLINE       0     0     0
              wwn-0x5000c5007a41d223              ONLINE       0     0     0
              ata-WDC_WD101EMAZ-11G7DA0_VCG9YGZM  ONLINE       0     0     0
            wwn-0x5000c5007a4232c4                ONLINE       0     0     0
            wwn-0x5000c5007a424ed9                ONLINE       0     0     0
            wwn-0x5000c5007a476cc0                ONLINE       0     0     0

errors: No known data errors

Commands for the next disks:

Command to resize the pool after all disks are replaced:

morey-tech commented 4 months ago

After replacing all of the disks, but before expanding the pool:

root@helios:~# zpool status storage-mass
  pool: storage-mass
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: resilvered 2.41T in 08:10:50 with 0 errors on Tue May 28 00:58:40 2024
config:

        NAME                                    STATE     READ WRITE CKSUM
        storage-mass                            ONLINE       0     0     0
          raidz2-0                              ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCG9YGZM  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGAM7KM  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGSDUGN  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCH2NUVP  ONLINE       0     0     0

errors: No known data errors
root@helios:~# zpool list storage-mass
NAME           SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
storage-mass  14.5T  10.1T  4.45T        -         -     6%    69%  1.00x    ONLINE  -

Expanding the pool:

root@helios:~# zpool online -e storage-mass ata-WDC_WD101EMAZ-11G7DA0_VCG9YGZM
root@helios:~# zpool online -e storage-mass ata-WDC_WD101EMAZ-11G7DA0_VCGAM7KM
root@helios:~# zpool online -e storage-mass ata-WDC_WD101EMAZ-11G7DA0_VCGSDUGN
root@helios:~# zpool online -e storage-mass ata-WDC_WD101EMAZ-11G7DA0_VCH2NUVP
root@helios:~# zpool list storage-mass
NAME           SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
storage-mass  36.4T  10.1T  26.3T        -         -     2%    27%  1.00x    ONLINE  -
morey-tech commented 4 months ago