search

khuedoan / homelab

Fully automated homelab from empty disk to running services with a single command.
https://homelab.khuedoan.com
GNU General Public License v3.0
8.25k stars 768 forks source link

Bare-metal provisioning #14

Closed ionut-maxim closed 2 years ago

ionut-maxim commented 2 years ago

Hey, this is just a proposal/question.

Have you considered using tinkerbell for bare-metal? I like the solution you have right now, but, I was wondering if there is a specific reason you didn't choose any of the existing solutions out there already.

khuedoan commented 2 years ago

Yes I did consider other options, but they have the same limitation: they require a preconfigured PXE server.

Let's take matchbox as an example, to run Terraform and boot your machines, you'll need to set up a matchbox server:

  1. Turn the matchbox server on
  2. Install Linux on it (probably via a USB)
  3. Install matchbox on it
  4. Install something for Terraform state backend like Minio
  5. Get the matchbox server + Terraform state backend address and the credentials, fill them in to *.tfvars (or export environment variables) to configure the Terraform provider

3 to 4 (and maybe 5) can be automated with a golden image, but you still have a special snowflake, and you lost one of your machine for the PXE server.

Only then you can run Terraform apply, meanwhile my setup just need a single make command and it will start the ephemeral PXE server, Wake-on-LAN, and wait for the servers to be ready.

I also experimented with Cluster API (Sidero), but I haven't been able to automate it to the level that I desired because of some missing features for consumer hardware (without BMC), and I'm still waiting for its prime time to return to it.

ionut-maxim commented 2 years ago

Hey, thanks for the response.

I was also looking into Sidero but I don't quite like the fact that it needs a k8s cluster already running. Ofcourse they have examples of quickly spinning up a local cluster but still.

khuedoan commented 2 years ago

Actually it's not that different from my setup, instead of an ephemeral PXE server, you can create an ephemeral cluster like this and install Sidero components there.

ionut-maxim commented 2 years ago

Again, thank you for your answers. Am going to keep a close 👁️ on your repo. Hopefully you reach stable soon 🎉

ionut-maxim commented 2 years ago

Ow, I just found out that longhorn-storage isn't supported on talos yet.

khuedoan commented 2 years ago

yeah you'll need something without iSCSI like Rook. BTW Mayastor just released v1.0.0