egress-selector-mode using flannel over wireguard

[jr200]

Environmental Info: k3s version v1.24.1+k3s1 (0581808f) go version go1.18.1

Node(s) CPU architecture, OS, and Version: Linux myserver 5.11.0-49-generic #55-Ubuntu SMP Wed Jan 12 17:36:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: 2 servers, 6 agents (but can reproduce on just 1 server.)

Describe the bug: kubectl logs -n kube-system coredns-b96499967-h4c5w Error from server: Get "https://myserver.k3s:10250/containerLogs/kube-system/coredns-b96499967-h4c5w/coredns": dial tcp 127.0.0.1:6443: connect: connection refused

• the above worked in v1.23.6+k3s1

Steps To Reproduce:

• I have networked my machines together using a wireguard mesh (via netmaker).
• k3s is installed on top of the wireguard network, i.e.,
  • I set flannel-iface: nm-k3s (my wireguard NIC), and set node-ips to be in that subnet
• (my laptop connects to my servers over vpn/wireguard as an external client)
• a fresh install of k3s v1.24.1+k3s1 reproduces the above.
• if I set egress-selector-mode: disabled, then the kubectl logs command works again.
• if i explicitly setegress-selector-mode to pod/agent/cluster, the logs commands fails as above.

Expected behavior: kubectl logs <container-id> should show the logs of the container.

Actual behavior: Error from server: Get "https://myserver.k3s:10250/containerLogs/kube-system/coredns-b96499967-h4c5w/coredns": dial tcp 127.0.0.1:6443: connect: connection refused

[brandond]

dial tcp 127.0.0.1:6443: connect: connection refused

This is somewhat unexpected. I suspect that you've customized the bind-address, and we probably need to account for that in the egress proxy config.

[rancher-max]

I can't recreate this on v1.24.1+k3s1 with only egress-selector-mode set, so I suspect Brad is correct that bind-address was also set. In that case, I'm going to close this as validated per https://github.com/k3s-io/k3s/issues/5712#issuecomment-1175659021 but feel free to reopen if we are mistaken.

[consideRatio]

Linking out to https://github.com/k3s-io/k3s/issues/5633 where an issue like this is still intermittently around as confirmed on many different k3s installations.