Closed danmanners closed 1 year ago
@danmanners looking at the logs, if you're re-using machine configuration across different clusters, this confuses cluster discovery and KubeSpan. It is important to use fresh secrets (.cluster.id
) for each cluster, as Discovery Service might keep stale discovery date (from previous cluster) up to 30 minutes.
@danmanners looking at the logs, if you're re-using machine configuration across different clusters, this confuses cluster discovery and KubeSpan. It is important to use fresh secrets (
.cluster.id
) for each cluster, as Discovery Service might keep stale discovery date (from previous cluster) up to 30 minutes.
Hey @smira, thanks for the reply. Unless I'm doing something incorrectly (which is entirely and highly possible), this is all for a single cluster in my homelab. The reason I'm attempting to use KubeSpan is because I will be spinning up nodes in AWS and Azure, and wanted to evaluate how well KubeSpan works for that use case. If that's the incorrect way to do things, I can try this again with KubeSpan disabled.
Thanks!
@danmanners I'm talking about the KubeSpan warning in the logs about node IP overlap. So there are two possibilities:
If you actually have duplicate node IPs, which I doubt, that is not supported with Kubespan.
For the stale discovery data, see my comment above, if you generate a machine config, then spin up a cluster, destroy it, and spin up another cluster from the same machine config, Discovery Service will keep data from the previous cluster for up to 30 minutes, and that might lead to the error above. Workaround is to generate fresh machine config for each cluster you're deploying (even if the previous one is destroyed).
Stale
No longer necessary. Closing.
Bug Report
While attempting to provision a cluster in Proxmox v7.X, the first control plane node can be deployed successfully, but any and all additional control plane or worker nodes will almost never connect, and eventually kernel panic and require a hard reset in order to attempt and fail connection again.
Description
Only a single master ever provisions successfully, and all other nodes refuse to connect.
I am deploying all of this on a Proxmox Cluster, and I have successfully deployed Talos nearly identically in the past (without KubeSpan) on version 0.13.X. I'm quite stumped at this point, and I'm trying to identify if I'm actually encountering an unexpected behavior, or if my configs are incorrect.
Conversely, I have been able to deploy nodes inconsistently. On occasion, all three control plane nodes will connect with each other and then kernel panic/crash even when connected to each other. I've re-deployed the VMs several times following the docs here to no avail.
Proxmox Screenshot
Control Plane Node Crashing Screenshot
Logs - Working Control Plane
Logs - Broken Worker
Environment
Talos version:
Kubernetes version:
Platform: Proxmox 7.x
Sanitized Control Plane Config - FUNCTIONAL
Sanitized Worker Node Config - NON-FUNCTIONAL
Additional Notes
Happy to provide any and all other requested information.