siderolabs / terraform-provider-talos

Mozilla Public License 2.0
123 stars 17 forks source link

BUG: Terraform defaulting to Talos v1.5.0 image #152

Closed Ramshield closed 7 months ago

Ramshield commented 7 months ago

Hi,

I am using the following Terraform code:

## Proxmox VM

resource "proxmox_vm_qemu" "talos_control_plane" {
  count = var.num_control_plane
  name  = "${var.cluster_name}-control-plane-${count.index}"

  target_node = var.proxmox_host

  agent    = 1
  cores    = 4
  sockets  = 1
  cpu      = "host"
  memory   = 4096
  scsihw   = "virtio-scsi-pci"
  bootdisk = "scsi0"
  #iso      = "local:iso/metal-amd64-v1.6.6_qemu_ifname_biosdevname.iso"
  pxe              = true
  boot             = "order=scsi0;net0"
  onboot           = true
  automatic_reboot = true

  disks {
    scsi {
      scsi0 {
        disk {
          storage = "nvme_pool"
          size    = 10
        }
      }
    }
  }

  network {
    model  = "virtio"
    bridge = "vmbr0"
  }

  lifecycle {
    ignore_changes = [
      network,
    ]
  }
}

## Talos

resource "talos_machine_secrets" "machine_secrets" {
  talos_version = "v1.6.6"
}

data "talos_client_configuration" "talosconfig" {
  cluster_name         = var.cluster_name
  client_configuration = talos_machine_secrets.machine_secrets.client_configuration
  endpoints            = proxmox_vm_qemu.talos_control_plane[*].default_ipv4_address
}

data "talos_machine_configuration" "machineconfig_cp" {
  cluster_name     = var.cluster_name
  cluster_endpoint = "https://10.0.0.9:8443"
  machine_type     = "controlplane"
  machine_secrets  = talos_machine_secrets.machine_secrets.machine_secrets
  talos_version    = "v1.6.6"
  #kubernetes_version = "v1.28.7"
}

resource "talos_machine_configuration_apply" "cp_config_apply" {
  client_configuration        = talos_machine_secrets.machine_secrets.client_configuration
  machine_configuration_input = data.talos_machine_configuration.machineconfig_cp.machine_configuration
  count                       = length(proxmox_vm_qemu.talos_control_plane)
  node                        = proxmox_vm_qemu.talos_control_plane[count.index].default_ipv4_address
}

resource "talos_machine_bootstrap" "bootstrap" {
  client_configuration = talos_machine_secrets.machine_secrets.client_configuration
  node                 = proxmox_vm_qemu.talos_control_plane[0].default_ipv4_address
}

data "talos_cluster_kubeconfig" "kubeconfig" {
  client_configuration = talos_machine_secrets.machine_secrets.client_configuration
  node                 = proxmox_vm_qemu.talos_control_plane[0].default_ipv4_address
}

This boots Talos v1.6.6 but then uses the installer image of v1.5.0, without my BIOS parameters and without the qemu-guest-agent extension. If it doesn't matter if I set either talos_version in talos_machine_secrets, or talos_machine_configuration. The result is the same, it always installs a default Talos v1.5.0 version.

What am I missing? Thank you!

Ramshield commented 7 months ago

This was with release version 0.3.2 of Terraform Talos module. I tried version 0.4.0, this installs v1.6.0, still not the version I specify, nor the Kubernetes version. Please advise.

UnstoppableMango commented 7 months ago

I'm not sure if this is the intended flow, but I believe the resulting Talos version depends on the version of the installer image. You could try setting machine.install.image to the version you need via a config patch.

UnstoppableMango commented 7 months ago

If you've made modifications such as kernel arguments or system extensions like qemu-guest-agent via Image Factory or other, I think setting machine.install.image is required. This seems to line up with how step 3 of the docs are worded

Ensure that machine configuration field .machine.install.image points to the custom installer image.

frezbo commented 7 months ago

You'd need to set .machine.install.image otherwise it defaults to whatever talos api version the provider uses

Ramshield commented 7 months ago

Oh I see @UnstoppableMango and @frezbo , I totally missed that. Does anyone of you perhaps have an example on how to set this with Terraform (I assume with config patches? I can't find an example and I am terrible with Terraform.

Thank you!

frezbo commented 7 months ago

yes, use a patch

Ramshield commented 7 months ago
data "talos_machine_configuration" "machineconfig_cp" {
  cluster_name     = var.cluster_name
  cluster_endpoint = "https://10.0.0.9:8443"
  machine_type     = "controlplane"
  machine_secrets  = talos_machine_secrets.machine_secrets.machine_secrets
  talos_version    = "v1.6.6"
  kubernetes_version = "v1.28.7"
  config_patches = [
    file("${path.module}/talos-patch.yaml")
  ]
}

Content of talos-patch.yaml:

---
machine:
  install:
    image: <INSTALLER URL FROM factory talos>

That did the trick!