search

kubernetes-sigs / kubespray

Deploy a Production Ready Kubernetes Cluster
Apache License 2.0
16.15k stars 6.47k forks source link

Wireguard not working #10538

Closed anutator closed 7 months ago

anutator commented 1 year ago

Kubespray 2.23.0, Cilium network plugin.

I can't enable IPSec encryption because it doesn't work with cilium_kube_proxy_replacement: strict (current limitation). So I try to use Wireguard type of encryption:

cilium_version: "v1.13.8"  # or 1.13.4 (the same result)
cilium_cli_version: "v0.15.11"
...
cilium_kube_proxy_replacement: strict
cilium_encryption_enabled: true
cilium_encryption_type: "wireguard"

cilium pods in crashLoopBackOff state:

cilium-pmhhg                                  0/1     CrashLoopBackOff    6 (4m33s ago)   11m
cilium-wh2mr                                  0/1     CrashLoopBackOff    6 (4m52s ago)   11m
cilium-zhwp2                                  0/1     CrashLoopBackOff    6 (4m31s ago)   11m

Errors in logs show that Wireguard is not compatible with L7 proxy:

level=info msg="Using autogenerated IPv4 allocation range" subsys=node v4Prefix=10.12.0.0/16
level=info msg=Invoked duration=1.303056ms function="gops.registerGopsHooks (cell.go:39)" subsys=hive
level=fatal msg="Wireguard (--enable-wireguard) is not compatible with L7 proxy (--enable-l7-proxy)" subsys=daemon

Related issues: https://github.com/kubernetes-sigs/kubespray/pull/9689 https://github.com/cilium/cilium/issues/15462 https://github.com/cilium/cilium/pull/19401

It seems that problem is resolved in cilium version 1.14+ but if I change cilium to 1.14.2 or 1.14.3 I get errors in cilium-operator pods:

cilium_version: "v1.14.3"
cilium_cli_version: "v0.15.11"
...
cilium_kube_proxy_replacement: strict
cilium_encryption_enabled: true
cilium_encryption_type: "wireguard"

Errors:

$ k logs cilium-operator-f9cfc6fb5-2cmcm
level=fatal msg="Option --routing-mode cannot be used in combination with --tunnel" subsys=config

Kubernetes 1.27.4 on our own virtual machines on Rocky Linux 9 installed using Kubespray 2.23.0

# cat /etc/*release
NAME="Rocky Linux"
VERSION="9.2 (Blue Onyx)"
ID="rocky"

$ uname -r
5.14.0-162.18.1.el9_1.x86_64

Rocky Linux 9.2 (Blue Onyx)   5.14.0-162.18.1.el9_1.x86_64   containerd://1.7.3

$ git rev-parse --short HEAD
e1558d2

Besides I didn't understand If I should install wireguard packages in Rocky Linux manually or not??? My kernel version is 5.14.0, but I have no wireguard-tools and kmod-wireguard. If I need this packages why they are not installed automatically using Ansible in Kubespray? In calico role a task "Calico | Install Wireguard packages" installs packages from variable calico_wireguard_packages, but in Cilium role I didn't find the same task. Should I install this packages manually?

sudo dnf install epel-release elrepo-release -y
sudo yum install kmod-wireguard wireguard-tools
k8s-triage-robot commented 9 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 8 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 7 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 7 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/kubespray/issues/10538#issuecomment-2028289609): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.