search

morey-tech / homelab

0 stars 0 forks source link

Provision helios (DL380 G9) to replace phobos #15

Closed morey-tech closed 4 months ago

morey-tech commented 7 months ago

The DL380 G9 has more memory (384 GB (12x 32GB) DDR4), better CPUs (E5-2650 v4), more PCI-e slots available, and better storage. Provision the server to replace phobos as the primary VM host.

morey-tech commented 7 months ago

The host could support a GPU for video transcoding and running AI models. Requires 10 pin to 6+8 pin power cable.

morey-tech commented 7 months ago

Errors during boot:

207-Memory initialization error on Processor 2, DIMM 9.
The operating system may not have access to all of the memory installed in the system.

207-Memory initialization error on Processor 2, DIMM 8.
The operating system may not have access to all of the memory Installed in the system.

Only 320 GB of memory is available, out of 384 GB total.

morey-tech commented 7 months ago

The host has an internal SD card of 64 GB (59.62 GiB), using that for proxmox install.

morey-tech commented 7 months ago
morey-tech commented 7 months ago

Helios has two 10Gb/s ports that use RJ45. These should connect to the core switch, but it uses QFP+ ports. I will need to get a transceiver to convert the ports on the switch. For now, only the 1Gb/s and iLO2 port is connected.

morey-tech commented 7 months ago

Added the SSD as a storage pool on proxmox.

pvcreate /dev/sda
vgcreate storage-ssd /dev/sda 
lvcreate -l +100%FREE -n pve storage-ssd
pvesm add lvmthin store-ssd --thinpool pve --vgname storage-ssd

I removed the default storage pool on the SD card and extended the root lv.

lvextend -l +100%FREE /dev/pve/root

The lvm storage backend only supports the images content type. I still need to use dir instead.

mkfs.ext4 /dev/storage-ssd/pve
mount /dev/storage-ssd/pve /mnt/storage-ssd/
pvesm add dir storage-ssd --path /mnt/storage-ssd
pvesm set storage-ssd --content images,rootdir,vztmpl,backup,iso,snippets

I also disabled all content types on the SD card storage pool.

pvesm set local --content none

Add the 1TB nvme drive in a new pool (and volume group):

pvcreate /dev/nvme1n1
vgcreate storage-nvme /dev/nvme1n1
lvcreate -l +100%FREE -n pve storage-nvme
mkfs.ext4 /dev/storage-nvme/pve
mkdir /mnt/storage-nvme
mount /dev/storage-nvme/pve /mnt/storage-nvme/
pvesm add dir storage-nvme --path /mnt/storage-nvme
pvesm set storage-nvme --content images,rootdir,vztmpl,backup,iso,snippets
morey-tech commented 7 months ago

Removed the enterprise deb repos and replaced them with the no-subscription repos.

# Proxmox VE pve-no-subscription repository provided by proxmox.com,
deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription
deb http://download.proxmox.com/debian/ceph-quincy bookworm no-subscription

https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_no_subscription_repo

morey-tech commented 4 months ago

How to move guests (VMs, CTs) from phobos to helios:

morey-tech commented 4 months ago 
root@helios:~# zpool import storage-media
cannot import 'storage-media': pool was previously in use from another system.
Last accessed by phobos (hostid=2b11b071) at Sun May 26 01:04:17 2024
The pool can be imported, use 'zpool import -f' to import the pool.

root@helios:~# zpool import -f storage-media
root@helios:~# zpool import -f storage-mass

root@helios:~# zpool list
NAME               SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
storage-mass      14.5T  10.1T  4.45T        -         -     6%    69%  1.00x    ONLINE  -
storage-mass-new  36.4T  1.17M  36.4T        -         -     0%     0%  1.00x    ONLINE  -
storage-media     72.8T  26.5T  46.2T        -         -     0%    36%  1.00x    ONLINE  -