search

vitobotta / hetzner-k3s

The easiest and fastest way to create and manage Kubernetes clusters in Hetzner Cloud using the lightweight distribution k3s by Rancher.
MIT License
1.77k stars 129 forks source link

2.0.0 - New cluster deploy fails #413

Closed rmelilloii closed 3 weeks ago

rmelilloii commented 3 weeks ago

Hello, good morning and happy Monday!

My deployment is failing without a clear reason.

My deploy.yaml

---
hetzner_token: ***
cluster_name: romulotest2
kubeconfig_path: "./kubeconfig"
k3s_version: v1.26.7+k3s1

networking:
  ssh:
    port: 22
    use_agent: false # set to true if your key has a passphrase
    public_key_path: "/home/pingu/.ssh/id_rmii.pub"
    private_key_path: "/home/pingu/.ssh/id_rmii"
  allowed_networks:
    ssh:
      - ***.***.90.54/32
    api: # this will firewall port 6443 on the nodes; it will NOT firewall the API load balancer
      - ***.***.90.54/32
  public_network:
    ipv4: true
    ipv6: true
  private_network:
    enabled : true
    subnet: 10.0.0.0/16
  cni:
    enabled: true
    encryption: false
    mode: flannel

datastore:
  mode: etcd # etcd (default) or external

schedule_workloads_on_masters: true

masters_pool:
  instance_type: cx32
  instance_count: 3
  image: ubuntu-24.04
  location: nbg1

worker_node_pools:
- name: pool-01
  instance_type: cx32
  instance_count: 1
  image: ubuntu-24.04
  location: nbg1

embedded_registry_mirror:
  enabled: true

Log:

pingu@RMII-DESK:~/hetznerDeploy2/romuloPlayground$ hetzner-k3s create --config ./hetzner-deploy.yaml
[Configuration] Validating configuration...
[Configuration] ...configuration seems valid.
[Private Network] Creating private network...
[Private Network] ...private network created
[SSH key] Creating SSH key...
[SSH key] ...SSH key created
[Placement groups] Creating placement group romulotest2-masters...
[Placement groups] ...placement group romulotest2-masters created
[Placement groups] Creating placement group romulotest2-pool-01-2...
[Placement groups] ...placement group romulotest2-pool-01-2 created
[Instance romulotest2-master1] Creating instance romulotest2-master1 (attempt 1)...
[Instance romulotest2-master3] Creating instance romulotest2-master3 (attempt 1)...
[Instance romulotest2-master2] Creating instance romulotest2-master2 (attempt 1)...
[Instance romulotest2-master1] Instance status: starting
[Instance romulotest2-master1] Powering on instance (attempt 1)
[Instance romulotest2-master1] Waiting for instance to be powered on...
[Instance romulotest2-master3] Instance status: off
[Instance romulotest2-master3] Powering on instance (attempt 1)
[Instance romulotest2-master3] Waiting for instance to be powered on...
[Instance romulotest2-master2] Instance status: starting
[Instance romulotest2-master2] Powering on instance (attempt 1)
[Instance romulotest2-master2] Waiting for instance to be powered on...
[Instance romulotest2-master1] Instance status: running
[Instance romulotest2-master3] Instance status: running
[Instance romulotest2-master2] Instance status: running
[Instance romulotest2-master1] Waiting for successful ssh connectivity with instance romulotest2-master1...
[Instance romulotest2-master3] Waiting for successful ssh connectivity with instance romulotest2-master3...
[Instance romulotest2-master2] Waiting for successful ssh connectivity with instance romulotest2-master2...
[Instance romulotest2-master3] ...instance romulotest2-master3 is now up.
[Instance romulotest2-master3] ...instance romulotest2-master3 created
[Instance romulotest2-master1] ...instance romulotest2-master1 is now up.
[Instance romulotest2-master1] ...instance romulotest2-master1 created
[Instance romulotest2-master2] ...instance romulotest2-master2 is now up.
[Instance romulotest2-master2] ...instance romulotest2-master2 created
[Firewall] Creating firewall...
[Firewall] ...firewall created
[Instance romulotest2-pool-pool-01-worker1] Creating instance romulotest2-pool-pool-01-worker1 (attempt 1)...
[Instance romulotest2-master3] 🕒 Awaiting cloud config (may take a minute...)
[Instance romulotest2-pool-pool-01-worker1] Instance status: starting
[Instance romulotest2-pool-pool-01-worker1] Powering on instance (attempt 1)
[Instance romulotest2-pool-pool-01-worker1] Waiting for instance to be powered on...
[Instance romulotest2-master3] .
[Instance romulotest2-master3] Cloud init finished: 35.42 - Mon, 19 Aug 2024 08:37:13 +0000 - v. 24.1.3-0ubuntu3.3
[Instance romulotest2-master3] [INFO]  Using v1.26.7+k3s1 as release
[Instance romulotest2-master3] [INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.26.7+k3s1/sha256sum-amd64.txt
[Instance romulotest2-master3] [INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.26.7+k3s1/k3s
[Instance romulotest2-master3] [INFO]  Verifying binary download
[Instance romulotest2-master3] [INFO]  Installing k3s to /usr/local/bin/k3s
[Instance romulotest2-master3] [INFO]  Skipping installation of SELinux RPM
[Instance romulotest2-master3] [INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[Instance romulotest2-master3] [INFO]  Creating /usr/local/bin/crictl symlink to k3s
[Instance romulotest2-master3] [INFO]  Creating /usr/local/bin/ctr symlink to k3s
[Instance romulotest2-master3] [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[Instance romulotest2-master3] [INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[Instance romulotest2-master3] [INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[Instance romulotest2-master3] [INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[Instance romulotest2-master3] [INFO]  systemd: Enabling k3s unit
[Instance romulotest2-master3] [INFO]  systemd: Starting k3s
[Instance romulotest2-master3] Waiting for the control plane to be ready...
[Instance romulotest2-pool-pool-01-worker1] Instance status: running
[Instance romulotest2-pool-pool-01-worker1] Waiting for successful ssh connectivity with instance romulotest2-pool-pool-01-worker1...
[Control plane] Generating the kubeconfig file to /home/pingu/hetznerDeploy2/romuloPlayground/kubeconfig...
error: no context exists with the name: "romulotest2-master3"
[Control plane] ...kubeconfig file generated as /home/pingu/hetznerDeploy2/romuloPlayground/kubeconfig.
[Instance romulotest2-pool-pool-01-worker1] ...instance romulotest2-pool-pool-01-worker1 is now up.
[Instance romulotest2-pool-pool-01-worker1] ...instance romulotest2-pool-pool-01-worker1 created
Unhandled exception in spawn: timeout after 00:00:30 (Tasker::Timeout)
  from /usr/lib/crystal/core/channel.cr:453:10 in 'timeout'
  from /home/runner/work/hetzner-k3s/hetzner-k3s/src/kubernetes/installer.cr:124:7 in 'run'
  from /usr/lib/crystal/core/fiber.cr:143:11 in 'run'
  from ???

The error occurs at around 3 min from start. And The maximum time I waited was 15 min. The log didn't change. The VMs and network are deployed but Kubernetes is not deployed on any node.

Trying to deploy the cluster using the latest k3s "v1.30.3k3s1", changes the outcome a little bit. Kubernetes is now deployed on the master1 node. Log:

[Configuration] Validating configuration...
[Configuration] ...configuration seems valid.
[Private Network] Creating private network...
[Private Network] ...private network created
[SSH key] Creating SSH key...
[SSH key] ...SSH key created
[Placement groups] Creating placement group romulotest2-masters...
[Placement groups] ...placement group romulotest2-masters created
[Placement groups] Creating placement group romulotest2-pool-01-2...
[Placement groups] ...placement group romulotest2-pool-01-2 created
[Instance romulotest2-master1] Creating instance romulotest2-master1 (attempt 1)...
[Instance romulotest2-master3] Creating instance romulotest2-master3 (attempt 1)...
[Instance romulotest2-master2] Creating instance romulotest2-master2 (attempt 1)...
[Instance romulotest2-master1] Instance status: starting
[Instance romulotest2-master1] Powering on instance (attempt 1)
[Instance romulotest2-master1] Waiting for instance to be powered on...
[Instance romulotest2-master3] Instance status: starting
[Instance romulotest2-master3] Powering on instance (attempt 1)
[Instance romulotest2-master3] Waiting for instance to be powered on...
[Instance romulotest2-master2] Instance status: off
[Instance romulotest2-master2] Powering on instance (attempt 1)
[Instance romulotest2-master2] Waiting for instance to be powered on...
[Instance romulotest2-master1] Instance status: running
[Instance romulotest2-master3] Instance status: running
[Instance romulotest2-master2] Instance status: running
[Instance romulotest2-master1] Waiting for successful ssh connectivity with instance romulotest2-master1...
[Instance romulotest2-master3] Waiting for successful ssh connectivity with instance romulotest2-master3...
[Instance romulotest2-master2] Waiting for successful ssh connectivity with instance romulotest2-master2...
[Instance romulotest2-master1] ...instance romulotest2-master1 is now up.
[Instance romulotest2-master1] ...instance romulotest2-master1 created
[Instance romulotest2-master3] ...instance romulotest2-master3 is now up.
[Instance romulotest2-master3] ...instance romulotest2-master3 created
[Instance romulotest2-master2] ...instance romulotest2-master2 is now up.
[Instance romulotest2-master2] ...instance romulotest2-master2 created
[Firewall] Creating firewall...
[Firewall] ...firewall created
[Instance romulotest2-pool-pool-01-worker1] Creating instance romulotest2-pool-pool-01-worker1 (attempt 1)...
[Instance romulotest2-master1] 🕒 Awaiting cloud config (may take a minute...)
[Instance romulotest2-pool-pool-01-worker1] Instance status: starting
[Instance romulotest2-pool-pool-01-worker1] Powering on instance (attempt 1)
[Instance romulotest2-pool-pool-01-worker1] Waiting for instance to be powered on...
[Instance romulotest2-master1] .
[Instance romulotest2-master1] Cloud init finished: 42.54 - Mon, 19 Aug 2024 08:49:16 +0000 - v. 24.1.3-0ubuntu3.3
[Instance romulotest2-master1] [INFO]  Using v1.30.3+k3s1 as release
[Instance romulotest2-master1] [INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/sha256sum-amd64.txt
[Instance romulotest2-master1] [INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/k3s
[Instance romulotest2-master1] [INFO]  Verifying binary download
[Instance romulotest2-master1] [INFO]  Installing k3s to /usr/local/bin/k3s
[Instance romulotest2-master1] [INFO]  Skipping installation of SELinux RPM
[Instance romulotest2-master1] [INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[Instance romulotest2-master1] [INFO]  Creating /usr/local/bin/crictl symlink to k3s
[Instance romulotest2-master1] [INFO]  Creating /usr/local/bin/ctr symlink to k3s
[Instance romulotest2-master1] [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[Instance romulotest2-master1] [INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[Instance romulotest2-master1] [INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[Instance romulotest2-master1] [INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[Instance romulotest2-master1] [INFO]  systemd: Enabling k3s unit
[Instance romulotest2-pool-pool-01-worker1] Instance status: running
[Instance romulotest2-master1] [INFO]  systemd: Starting k3s
[Instance romulotest2-pool-pool-01-worker1] Waiting for successful ssh connectivity with instance romulotest2-pool-pool-01-worker1...
[Instance romulotest2-master1] Waiting for the control plane to be ready...
[Instance romulotest2-pool-pool-01-worker1] ...instance romulotest2-pool-pool-01-worker1 is now up.
[Instance romulotest2-pool-pool-01-worker1] ...instance romulotest2-pool-pool-01-worker1 created
[Control plane] Generating the kubeconfig file to /home/pingu/hetznerDeploy2/romuloPlayground/kubeconfig...
Switched to context "romulotest2-master1".
[Control plane] ...kubeconfig file generated as /home/pingu/hetznerDeploy2/romuloPlayground/kubeconfig.
[Instance romulotest2-master1] ...k3s deployed
[Instance romulotest2-master3] Cloud init finished: 38.19 - Mon, 19 Aug 2024 08:49:13 +0000 - v. 24.1.3-0ubuntu3.3
[Instance romulotest2-master2] Cloud init finished: 44.11 - Mon, 19 Aug 2024 08:49:23 +0000 - v. 24.1.3-0ubuntu3.3
[Instance romulotest2-master3] [INFO]  Using v1.30.3+k3s1 as release
[Instance romulotest2-master3] [INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/sha256sum-amd64.txt
[Instance romulotest2-master2] [INFO]  Using v1.30.3+k3s1 as release
[Instance romulotest2-master2] [INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/sha256sum-amd64.txt
[Instance romulotest2-master3] [INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/k3s
[Instance romulotest2-master2] [INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/k3s
[Instance romulotest2-master2] [INFO]  Verifying binary download
[Instance romulotest2-master2] [INFO]  Installing k3s to /usr/local/bin/k3s
[Instance romulotest2-master2] [INFO]  Skipping installation of SELinux RPM
[Instance romulotest2-master2] [INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[Instance romulotest2-master2] [INFO]  Creating /usr/local/bin/crictl symlink to k3s
[Instance romulotest2-master2] [INFO]  Creating /usr/local/bin/ctr symlink to k3s
[Instance romulotest2-master2] [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[Instance romulotest2-master2] [INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[Instance romulotest2-master2] [INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[Instance romulotest2-master2] [INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[Instance romulotest2-master2] [INFO]  systemd: Enabling k3s unit
[Instance romulotest2-master3] [INFO]  Verifying binary download
[Instance romulotest2-master3] [INFO]  Installing k3s to /usr/local/bin/k3s
[Instance romulotest2-master3] [INFO]  Skipping installation of SELinux RPM
[Instance romulotest2-master3] [INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[Instance romulotest2-master3] [INFO]  Creating /usr/local/bin/crictl symlink to k3s
[Instance romulotest2-master3] [INFO]  Creating /usr/local/bin/ctr symlink to k3s
[Instance romulotest2-master3] [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[Instance romulotest2-master3] [INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[Instance romulotest2-master3] [INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[Instance romulotest2-master3] [INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[Instance romulotest2-master3] [INFO]  systemd: Enabling k3s unit
[Instance romulotest2-master2] [INFO]  systemd: Starting k3s
[Instance romulotest2-master3] [INFO]  systemd: Starting k3s
[Instance romulotest2-master3] ...k3s deployed
[Instance romulotest2-master2] ...k3s deployed
[Control plane] Generating the kubeconfig file to /home/pingu/hetznerDeploy2/romuloPlayground/kubeconfig...
Switched to context "romulotest2-master1".
[Control plane] ...kubeconfig file generated as /home/pingu/hetznerDeploy2/romuloPlayground/kubeconfig.
[Hetzner Cloud Secret] Creating secret for Hetzner Cloud token...
[Hetzner Cloud Secret] secret/hcloud created
[Hetzner Cloud Secret] ...secret created
[Hetzner Cloud Controller] Installing Hetzner Cloud Controller Manager...
[Hetzner Cloud Controller] serviceaccount/hcloud-cloud-controller-manager created
[Hetzner Cloud Controller] clusterrolebinding.rbac.authorization.k8s.io/system:hcloud-cloud-controller-manager created
[Hetzner Cloud Controller] deployment.apps/hcloud-cloud-controller-manager created
[Hetzner Cloud Controller] Hetzner Cloud Controller Manager installed
[Hetzner CSI Driver] Installing Hetzner CSI Driver...
[Hetzner CSI Driver] serviceaccount/hcloud-csi-controller created
[Hetzner CSI Driver] storageclass.storage.k8s.io/hcloud-volumes created
[Hetzner CSI Driver] clusterrole.rbac.authorization.k8s.io/hcloud-csi-controller created
[Hetzner CSI Driver] clusterrolebinding.rbac.authorization.k8s.io/hcloud-csi-controller created
[Hetzner CSI Driver] service/hcloud-csi-controller-metrics created
[Hetzner CSI Driver] service/hcloud-csi-node-metrics created
[Hetzner CSI Driver] daemonset.apps/hcloud-csi-node created
[Hetzner CSI Driver] deployment.apps/hcloud-csi-controller created
[Hetzner CSI Driver] csidriver.storage.k8s.io/csi.hetzner.cloud created
[Hetzner CSI Driver] Hetzner CSI Driver installed
[System Upgrade Controller] Installing System Upgrade Controller...
[System Upgrade Controller] namespace/system-upgrade created
[System Upgrade Controller] customresourcedefinition.apiextensions.k8s.io/plans.upgrade.cattle.io created
[System Upgrade Controller] clusterrole.rbac.authorization.k8s.io/system-upgrade-controller created
[System Upgrade Controller] role.rbac.authorization.k8s.io/system-upgrade-controller created
[System Upgrade Controller] clusterrole.rbac.authorization.k8s.io/system-upgrade-controller-drainer created
[System Upgrade Controller] clusterrolebinding.rbac.authorization.k8s.io/system-upgrade-drainer created
[System Upgrade Controller] clusterrolebinding.rbac.authorization.k8s.io/system-upgrade created
[System Upgrade Controller] rolebinding.rbac.authorization.k8s.io/system-upgrade created
[System Upgrade Controller] namespace/system-upgrade configured
[System Upgrade Controller] serviceaccount/system-upgrade created
[System Upgrade Controller] configmap/default-controller-env created
[System Upgrade Controller] deployment.apps/system-upgrade-controller created
[System Upgrade Controller] ...System Upgrade Controller installed
[Cluster Autoscaler] Installing Cluster Autoscaler...
[Cluster Autoscaler] serviceaccount/cluster-autoscaler created
[Cluster Autoscaler] clusterrole.rbac.authorization.k8s.io/cluster-autoscaler created
[Cluster Autoscaler] role.rbac.authorization.k8s.io/cluster-autoscaler created
[Cluster Autoscaler] clusterrolebinding.rbac.authorization.k8s.io/cluster-autoscaler created
[Cluster Autoscaler] rolebinding.rbac.authorization.k8s.io/cluster-autoscaler created
[Cluster Autoscaler] deployment.apps/cluster-autoscaler created
[Cluster Autoscaler] ...Cluster Autoscaler installed
[Instance romulotest2-pool-pool-01-worker1] Cloud init finished: 42.02 - Mon, 19 Aug 2024 08:49:59 +0000 - v. 24.1.3-0ubuntu3.3
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Using v1.30.3+k3s1 as release
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/sha256sum-amd64.txt
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.30.3+k3s1/k3s
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Verifying binary download
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Installing k3s to /usr/local/bin/k3s
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Skipping installation of SELinux RPM
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Creating /usr/local/bin/crictl symlink to k3s
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Creating /usr/local/bin/ctr symlink to k3s
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[Instance romulotest2-pool-pool-01-worker1] [INFO]  Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[Instance romulotest2-pool-pool-01-worker1] [INFO]  env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[Instance romulotest2-pool-pool-01-worker1] [INFO]  systemd: Creating service file /etc/systemd/system/k3s-agent.service
[Instance romulotest2-pool-pool-01-worker1] [INFO]  systemd: Enabling k3s-agent unit
[Instance romulotest2-pool-pool-01-worker1] [INFO]  systemd: Starting k3s-agent

Should I try to enforce versions for the hetzner manifests?

vitobotta commented 3 weeks ago

Hi! In the first case I think the problem is that you have enabled the embedded registry mirror with v1.26.7+k3s1, which seems unsupported - see https://docs.k3s.io/installation/registry-mirror.

In 2.0.1 I will add a constraint/check to catch this kind of condition and abort telling the user why.

With the newer version of k3s, 1.30, did you try re-running the create command? If you do kubectl get nodes does worker 1 appear or not?

I'll do a quick test with your config now.

vitobotta commented 3 weeks ago

Hi, I found the problem. It's due to the firewall restriction. I'm about to implement a fix now.

rmelilloii commented 3 weeks ago

Fantastic! Thank you very much for that. Will it be under the same 2.0.0? I’ll do further tests as I do a bunch of other extra confit to my nodes and also enable k3s audit.

Cheers!

Romulo

On Mon, 19 Aug 2024 at 12:23, Vito Botta @.***> wrote:

Hi, I found the problem. It's due to the firewall restriction. I'm about to implement a fix now.

— Reply to this email directly, view it on GitHub https://github.com/vitobotta/hetzner-k3s/issues/413#issuecomment-2296338119, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG3HX5BNSFZHKN5WC75WZG3ZSHISDAVCNFSM6AAAAABMXMSQOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJWGMZTQMJRHE . You are receiving this because you authored the thread.Message ID: @.***>

vitobotta commented 3 weeks ago

I am releasing 2.0.1 with the fix as it's easier with a new version. Description of the problem:

In previous versions of the tool, a load balancer was created for the Kubernetes API when the cluster was HA (with multiple masters). Due to popular demand, I have removed this requirement so in v2, instead of creating the load balancer, a multi-context kubeconfig is generated to connect directly to any master without load balancer. This saves money on the load balancer and is more secure since access to the nodes can be restricted by IP address while this cannot be done with the load balancers.

The problem you encountered, and which I just fixed, is that the nodes were still using the public IP of the master(s) to set up k3s as agent, because that was the behavior with the load balancer. Now that the nodes are connecting directly to the master, they need to use the private IP if the private network is in use, since traffic in the private network is unrestricted and therefore even if you restrict access to the API from the Internet to specific addresses, this won't affect worker nodes.

v2.0.1 is now building - https://github.com/vitobotta/hetzner-k3s/actions/runs/10452534522

You can already test with the binary when it's available for your OS. I will update the brew formula for mac soon.

vitobotta commented 3 weeks ago

The new release is now ready. Please upgrade and try again. Thanks :)

rmelilloii commented 3 weeks ago

Brilliant, working perfectly now. I will proceed with all my extras and report back.

 Again, thank you and everyone involved in this project.

Romulo

On Mon, 19 Aug 2024 at 12:50, Vito Botta @.***> wrote:

The new release is now ready. Please upgrade and try again. Thanks :)

— Reply to this email directly, view it on GitHub https://github.com/vitobotta/hetzner-k3s/issues/413#issuecomment-2296386310, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG3HX5H5UIV2SL2DZTAF473ZSHLY3AVCNFSM6AAAAABMXMSQOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJWGM4DMMZRGA . You are receiving this because you authored the thread.Message ID: @.***>

vitobotta commented 3 weeks ago

Nice! Can we close this now? You can open separate issues if you run into other problems :)

rmelilloii commented 3 weeks ago

Please close yes.

Cheers.

Romulo

On Mon, 19 Aug 2024 at 13:07, Vito Botta @.***> wrote:

Nice! Can we close this now? You can open separate issues if you run into other problems :)

— Reply to this email directly, view it on GitHub https://github.com/vitobotta/hetzner-k3s/issues/413#issuecomment-2296416443, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG3HX5EXEX7K6SA5Q5FLCBLZSHNW7AVCNFSM6AAAAABMXMSQOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJWGQYTMNBUGM . You are receiving this because you authored the thread.Message ID: @.***>

vitobotta commented 3 weeks ago

Thanks