search

morey-tech / homelab

0 stars 0 forks source link

Add vdev with 8x 10TB drives to `storage-media` zpool #35

Closed morey-tech closed 2 months ago

morey-tech commented 2 months ago

Existing storage-media zpool:

root@helios:~# zpool status storage-media
  pool: storage-media
 state: ONLINE
config:

        NAME                                    STATE     READ WRITE CKSUM
        storage-media                           ONLINE       0     0     0
          raidz2-0                              ONLINE       0     0     0
            scsi-35000cca266d52aac              ONLINE       0     0     0
            scsi-35000cca266d48d60              ONLINE       0     0     0
            scsi-35000cca266a824cc              ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCH03P7P  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGEES4M  ONLINE       0     0     0
            scsi-35000cca266d5cb30              ONLINE       0     0     0
            scsi-35000cca266aa12e0              ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCG9WRLN  ONLINE       0     0     0

das2 contains 12 10TB drives, 8 of which should be used to expand the storage-media with a new raidz2 vdev.

To access the disks, das2 needs to be connected to helios, which depends on moving the new storage-mass disks (#34) from das3 replacing the 4TB disks in das1.

morey-tech commented 2 months ago

Hard drives by ids:

ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N
ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N
ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P
ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN
ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN
ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP
ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P
ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP

Found by grep for the serial number on the drive's front label from the disks by-id:

root@helios:~# ls -l /dev/disk/by-id | grep -i 0h50n
lrwxrwxrwx 1 root root  9 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N -> ../../sdu
lrwxrwxrwx 1 root root 10 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N-part1 -> ../../sdu1
lrwxrwxrwx 1 root root 10 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N-part9 -> ../../sdu9
root@helios:~# ls -l /dev/disk/by-id | grep -i 8j25n
lrwxrwxrwx 1 root root  9 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N -> ../../sds
lrwxrwxrwx 1 root root 10 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N-part1 -> ../../sds1
lrwxrwxrwx 1 root root 10 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N-part9 -> ../../sds9
root@helios:~# ls -l /dev/disk/by-id | grep -i pth0p
lrwxrwxrwx 1 root root 10 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P -> ../../sdad
lrwxrwxrwx 1 root root 11 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P-part1 -> ../../sdad1
lrwxrwxrwx 1 root root 11 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P-part9 -> ../../sdad9
root@helios:~# ls -l /dev/disk/by-id | grep -i 9v8m
lrwxrwxrwx 1 root root 10 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN -> ../../sdac
lrwxrwxrwx 1 root root 11 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN-part1 -> ../../sdac1
lrwxrwxrwx 1 root root 11 May 28 08:48 ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN-part9 -> ../../sdac9
root@helios:~# ls -l /dev/disk/by-id | grep -i ap9rn
lrwxrwxrwx 1 root root  9 May 28 08:50 ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN -> ../../sdn
root@helios:~# ls -l /dev/disk/by-id | grep -i 4k8mp
lrwxrwxrwx 1 root root  9 May 28 08:50 ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP -> ../../sdo
root@helios:~# ls -l /dev/disk/by-id | grep -i yxe5p
lrwxrwxrwx 1 root root  9 May 28 08:50 ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P -> ../../sdp
root@helios:~# ls -l /dev/disk/by-id | grep -i 0zryp
lrwxrwxrwx 1 root root  9 May 28 08:50 ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP -> ../../sdq

Confirmed each disk ID corresponds to a 10TB disk:

root@helios:~# disk_ids=(
  "ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N"
  "ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N"
  "ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P"
  "ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN"
  "ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN"
  "ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP"
  "ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P"
  "ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP"
)

for disk_id in "${disk_ids[@]}"; do
  echo "Disk ID: $disk_id"
  smartctl -i "/dev/disk/by-id/$disk_id" | grep "Capacity"
done
Disk ID: ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Disk ID: ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Disk ID: ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Disk ID: ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Disk ID: ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Disk ID: ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Disk ID: ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Disk ID: ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
morey-tech commented 2 months ago

Command to add vdev to pool (dry-run):

zpool add storage-media raidz2 -f \
    ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N \
    ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N \
    ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P \
    ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN \
    ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN \
    ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP \
    ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P \
    ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP -n

4 disks are from the previous Media pool, which is no longer in use. The command will need -f to add those drives. 

root@helios:~# zpool add storage-media raidz2 ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N     ata-WDC_WD101EMAZ-11
G7DA0_VCG8J25N     ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P     ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN     ata-WDC_
WD101EMAZ-11G7DA0_VCGAP9RN     ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP     ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P 
    ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP -n
invalid vdev specification
use '-f' to override the following errors:
/dev/disk/by-id/ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N-part1 is part of potentially active pool 'Media'
/dev/disk/by-id/ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N-part1 is part of potentially active pool 'Media'
/dev/disk/by-id/ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P-part1 is part of potentially active pool 'Media'
/dev/disk/by-id/ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN-part1 is part of potentially active pool 'Media'
morey-tech commented 2 months ago

Done!

root@helios:~# zpool add storage-media raidz2 -f     ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N     ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N     ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P     ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN     ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN     ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP     ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P     ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP -n
would update 'storage-media' to the following configuration:

        storage-media
          raidz2-0
            scsi-35000cca266d52aac
            scsi-35000cca266d48d60
            scsi-35000cca266a824cc
            ata-WDC_WD101EMAZ-11G7DA0_VCH03P7P
            ata-WDC_WD101EMAZ-11G7DA0_VCGEES4M
            scsi-35000cca266d5cb30
            scsi-35000cca266aa12e0
            ata-WDC_WD101EMAZ-11G7DA0_VCG9WRLN
          raidz2
            ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N
            ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N
            ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P
            ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN
            ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN
            ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP
            ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P
            ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP
root@helios:~# zpool add storage-media raidz2 -f     ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N     ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N     ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P     ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN     ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN     ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP     ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P     ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP
root@helios:~# zpool status storage-media
  pool: storage-media
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 09:28:51 with 0 errors on Sun May 12 09:52:56 2024
config:

        NAME                                    STATE     READ WRITE CKSUM
        storage-media                           ONLINE       0     0     0
          raidz2-0                              ONLINE       0     0     0
            scsi-35000cca266d52aac              ONLINE       0     0     0
            scsi-35000cca266d48d60              ONLINE       0     0     0
            scsi-35000cca266a824cc              ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCH03P7P  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGEES4M  ONLINE       0     0     0
            scsi-35000cca266d5cb30              ONLINE       0     0     0
            scsi-35000cca266aa12e0              ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCG9WRLN  ONLINE       0     0     0
          raidz2-1                              ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCH0H50N  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCG8J25N  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGPTH0P  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCG9V8MN  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGAP9RN  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCH4K8MP  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCGYXE5P  ONLINE       0     0     0
            ata-WDC_WD101EMAZ-11G7DA0_VCH0ZRYP  ONLINE       0     0     0

errors: No known data errors
root@helios:~# zpool list storage-media
NAME            SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
storage-media   146T  26.5T   119T        -         -     0%    18%  1.00x    ONLINE  -