search

cloyne / network

5 stars 5 forks source link

Server 3 issues #94

Closed mitar closed 7 years ago

mitar commented 7 years ago

In the past few months server 3 started having issues with hard drives. SMART is every so and so reporting that this or that hard drive has some issues, and dmesg is full of errors that RAID card had to be restarted. So I am not sure if issues are with hard drives, RAID card, drivers in general, or something else.

mitar commented 7 years ago

So one 750 GB drive (in bay 8) failed. I removed it from LVM. First, I resized LVM to make it smaller:

lvresize --resizefs --size -800G /dev/mapper/vg0-srv

Then I removed md4:

pvmove /dev/md4
vgreduce vg0 /dev/md4
pvremove /dev/md4
mdadm --stop /dev/md4
mdadm --zero-superblock /dev/sdg1

/dev/sdg1 can now be used as a replacement disk for any other 750 GB RAID-1 if any other disk fail. That one works. The other drive I pulled out.

Current state:

pvs -o+pv_used
  PV         VG   Fmt  Attr PSize   PFree   Used   
  /dev/md0   vg0  lvm2 a--    2.73t      0    2.73t
  /dev/md1   vg0  lvm2 a--    2.73t      0    2.73t
  /dev/md2   vg0  lvm2 a--    2.73t      0    2.73t
  /dev/md3   vg0  lvm2 a--    2.73t      0    2.73t
  /dev/md5   vg0  lvm2 a--  698.35g      0  698.35g
  /dev/md6   vg0  lvm2 a--  698.35g 101.65g 596.70g
  /dev/md7   vg0  lvm2 a--  698.35g      0  698.35g
lvdisplay /dev/mapper/vg0-srv
  --- Logical volume ---
  LV Path                /dev/vg0/srv
  LV Name                srv
  VG Name                vg0
  LV UUID                UvYIg3-QMId-m19Y-BeQ5-DtQV-QSPK-znMAFt
  LV Write Access        read/write
  LV Creation host, time server3, 2015-05-16 23:15:51 -0700
  LV Status              available
  # open                 1
  LV Size                12.86 TiB
  Current LE             3371192
  Segments               7
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:0