Latest version does not work with existing cluster and running " terraform apply -auto-approve"

[zypriafl]

Description

Hello,

Using the latest version we get the following error. For the moment we set "version = "v1.10.9"" as workaround.

terraform apply -auto-approve module.kube-hetzner.data.hcloud_servers.autoscaled_nodes["autoscaler"]: Reading... module.kube-hetzner.hcloud_network.k3s: Refreshing state... [id=2568275] module.kube-hetzner.hcloud_ssh_key.k3s[0]: Refreshing state... [id=10282504] module.kube-hetzner.data.hcloud_image.microos_snapshot: Reading... module.kube-hetzner.hcloud_placement_group.control_plane[0]: Refreshing state... [id=130759] module.kube-hetzner.hcloud_firewall.k3s: Refreshing state... [id=749497] module.kube-hetzner.data.github_release.kured[0]: Reading... module.kube-hetzner.data.github_release.hetzner_csi[0]: Reading... module.kube-hetzner.data.github_release.calico[0]: Reading... module.kube-hetzner.data.github_release.hetzner_ccm[0]: Reading... module.kube-hetzner.data.github_release.hetzner_csi[0]: Read complete after 0s [id=93149128] module.kube-hetzner.hcloud_network_subnet.agent[0]: Refreshing state... [id=2568275-172.16.0.0/24] module.kube-hetzner.hcloud_network_subnet.control_plane[0]: Refreshing state... [id=2568275-172.16.255.0/24] module.kube-hetzner.module.control_planes["0-0-control-plane-fsn1"].random_string.identity_file: Refreshing state... [id=kn5s6zqvtjee0apl6519] module.kube-hetzner.module.control_planes["0-0-control-plane-fsn1"].random_string.server: Refreshing state... [id=zbu] module.kube-hetzner.data.github_release.calico[0]: Read complete after 1s [id=88796454] module.kube-hetzner.module.control_planes["0-0-control-plane-fsn1"].data.cloudinit_config.config: Reading... module.kube-hetzner.module.control_planes["0-0-control-plane-fsn1"].data.cloudinit_config.config: Read complete after 0s [id=3924688064] module.kube-hetzner.data.github_release.hetzner_ccm[0]: Read complete after 1s [id=95988311] module.kube-hetzner.data.hcloud_servers.autoscaled_nodes["autoscaler"]: Read complete after 1s [id=9c2b069b3ce6f4e49f55a7ced66c187022aa0fe9] module.kube-hetzner.data.github_release.kured[0]: Read complete after 1s [id=95732764] ╷ │ Error: no image found for selector "microos-snapshot=yes" │ │ with module.kube-hetzner.data.hcloud_image.microos_snapshot, │ on .terraform/modules/kube-hetzner/main.tf line 6, in data "hcloud_image" "microos_snapshot": │ 6: data "hcloud_image" "microos_snapshot" {

Kube.tf file

module "kube-hetzner" {
  providers = {
    hcloud = hcloud
  }
  hcloud_token = local.hcloud_token
  source = "kube-hetzner/kube-hetzner/hcloud"
  ssh_public_key = file("/home/***/.ssh/web_app_scaler.pub")
  ssh_private_key = file("/home/***/.ssh/web_app_scaler")
  network_region = "eu-central"
  control_plane_nodepools = [
    {
      name        = "control-plane-fsn1",
      server_type = "cpx11",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 1
    }
  ]
  agent_nodepools = [
    {
      name        = "agent-middle",
      server_type = "cx31",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 0
    },
    {
      name        = "agent-large",
      server_type = "cx51",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 2
    },    
  ]
  extra_firewall_rules = [
    {
      description     = "For outgoing."
      direction       = "out"
      protocol        = "tcp"
      port            = "any"
      source_ips      = []
      destination_ips = ["0.0.0.0/0", "::/0"]
    }
  ]
  network_ipv4_cidr = "172.16.0.0/16"
  cluster_ipv4_cidr = "172.17.0.0/24"
  load_balancer_type     = "lb11"
  load_balancer_location = "fsn1"
  base_domain = "app.****.com"
  disable_hetzner_csi = true
  automatically_upgrade_k3s = false
  automatically_upgrade_os = false
  ingress_controller = "none"
  cluster_name = "app-scaler"
  allow_scheduling_on_control_plane = true
  lb_hostname = "app.*****.com"
  autoscaler_nodepools = [
    {
      name        = "autoscaler"
      server_type = "cx41"
      location    = "fsn1"
      min_nodes   = 0
      max_nodes   = 5
    }
  ]
}
provider "hcloud" {
  token = local.hcloud_token
}

terraform {
  required_version = ">= 1.3.3"
  required_providers {
    hcloud = {
      source  = "hetznercloud/hcloud"
      version = ">= 1.35.2"
    }
  }
}

output "kubeconfig" {
  value     = module.kube-hetzner.kubeconfig
  sensitive = true
}

Screenshots

No response

Platform

Linux

[jhelbling]

+

the same issue

[jhelbling]

Debug:

2023-03-29T13:01:54.588+0200 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/integrations/github/5.18.3/darwin_arm64/terraform-provider-github_v5.18.3 pid=27974 2023-03-29T13:01:54.588+0200 [DEBUG] provider: plugin exited 2023-03-29T13:01:54.588+0200 [TRACE] vertex "provider[\"registry.terraform.io/integrations/github\"] (close)": visit complete 2023-03-29T13:01:54.588+0200 [TRACE] dag/walk: upstream of "root" errored, so skipping 2023-03-29T13:01:54.588+0200 [INFO] backend/local: plan operation completed

[skhokhlov]

@zypriafl @jhelbling There is a new step need to be done before applying configuration.

[ifeulner]

Please read the docs. You have to create a snapshot image first.

[mysticaltech]

@zypriafl @jhelbling As @ifeulner said, please read the docs, the flow has changed a little bit and you now need an initial step.

[Lennix]

This also happened with another issue. Maybe you could add a section to the README explaining how to upgrade an existing cluster. @mysticaltech As it stands it says "Getting started" and "[Do not skip] Creating your kube.tf file and the OpenSUSE MicroOS snapshot" - which is not necessarily understood as upgrade instructions, especially since not everybody remembers what they did to create the cluster in the first place (if its several months ago).

[mysticaltech]

@Lennix We have an upgrade section in the Readme, and also an discussion announcement on the matter. However, if you see ways to improve, PRs are always welcome!

[Lennix]

You are absolutely right, I am sorry.