berops / claudie

Cloud-agnostic managed Kubernetes
https://docs.claudie.io/
Apache License 2.0
600 stars 41 forks source link

Bug: Kubeone fails to elect leader with proxy. #1515

Closed Despire closed 3 weeks ago

Despire commented 1 month ago

Current Behaviour

If the proxy is set and changes are made to an already existing cluster kubeone fails to elect the leader. Logs are provided below. After a certain amount of retries the leader election succeeds. With HTTP_PROXY off this does not occur.

time="16:17:28 CEST" level=info msg="Determine hostname..."
time="16:17:29 CEST" level=info msg="Determine operating system..."
time="16:17:29 CEST" level=info msg="Running host probes..."
time="16:17:32 CEST" level=info msg="Electing cluster leader..."
time="16:17:32 CEST" level=error msg="Failed to elect leader."
time="16:17:32 CEST" level=error msg="Quorum is mostly like lost, manual cluster repair might be needed."
time="16:17:32 CEST" level=error msg="Consider the KubeOne documentation for further steps."
time="16:17:32 CEST" level=warning msg="Task failed, error was: runtime: leader electing\nquorum mostly like lost"
time="16:17:42 CEST" level=warning msg="Retrying task..."
time="16:17:42 CEST" level=info msg="Running host probes..."
time="16:17:45 CEST" level=info msg="Electing cluster leader..."
time="16:17:45 CEST" level=error msg="Failed to elect leader."
time="16:17:45 CEST" level=error msg="Quorum is mostly like lost, manual cluster repair might be needed."
time="16:17:45 CEST" level=error msg="Consider the KubeOne documentation for further steps."
time="16:17:45 CEST" level=warning msg="Task failed, error was: runtime: leader electing\nquorum mostly like lost"
time="16:17:59 CEST" level=warning msg="Retrying task..."
time="16:17:59 CEST" level=info msg="Running host probes..."
time="16:18:02 CEST" level=info msg="Electing cluster leader..."
time="16:18:02 CEST" level=error msg="Failed to elect leader."
time="16:18:02 CEST" level=error msg="Quorum is mostly like lost, manual cluster repair might be needed."
time="16:18:02 CEST" level=error msg="Consider the KubeOne documentation for further steps."
time="16:18:02 CEST" level=warning msg="Task failed, error was: runtime: leader electing\nquorum mostly like lost"
time="16:18:21 CEST" level=warning msg="Retrying task..."
time="16:18:21 CEST" level=info msg="Running host probes..."
time="16:18:24 CEST" level=info msg="Electing cluster leader..."
time="16:18:24 CEST" level=info msg="Elected leader \"htz-ctrl-nodes-ec80tas-01\"..."

Expected Behaviour

with HTTP_PROXY enabled, ideally it should behave the same as with the HTTP_PROXY=off i.e. no need to have multiple retries before succeeding to elect a leader.

Steps To Reproduce

  1. Build cluster
  2. Change cluster by adding nodepools
  3. see kube-eleven logs
bernardhalas commented 1 month ago

Is being solved together with #1519