search

xiaods / k8e

K8E - Kubernetes Easy Engine
https://getk8e.com
Apache License 2.0
391 stars 25 forks source link

the cilium intalled fail on latest version #275

Closed snakeliwei closed 1 year ago

snakeliwei commented 1 year ago

this is the k8e log

Oct 10 17:13:17 VM-0-118-ubuntu k8e[11777]: E1010 17:13:17.030292 11777 pod_workers.go:965] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"metrics-server-74474969b-45hmf_kube-system(6e500cfa-1ea8-4097-8c75-ca6c551810d6)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\"metrics-server-74474969b-45hmf_kube-system(6e500cfa-1ea8-4097-8c75-ca6c551810d6)\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\"3580f149a703582240f49bfbc26ff54cf7d2e85012fda5547e632a369471a9f9\\": plugin type=\\"cilium-cni\\" name=\\"cilium\\" failed (add): unable to connect to Cilium daemon: failed to create cilium agent client after 30.000000 seconds timeout: Get \\"http:///var/run/cilium/cilium.sock/v1/config\\": dial unix /var/run/cilium/cilium.sock: connect: no such file or directory\nIs the agent running?\"" pod="kube-system/metrics-server-74474969b-45hmf" podUID=6e500cfa-1ea8-4097-8c75-ca6c551810d6

snakeliwei commented 1 year ago

ubuntu 22.04

xiaods commented 1 year ago

it seem ciilum 1.12.2 have this bug.

xiaods commented 1 year ago

@snakeliwei hi buddy,

Please do me a favor to testing the cilium install works:

cilium uninstall

cilium install --helm-set-string=ipam.operator.clusterPoolIPv4PodCIDR="10.42.0.0/16",kubeProxyReplacement=strict

for my ubuntu 22.04, it works. and k8s install.sh already update right now.

snakeliwei commented 1 year ago

It is work on master node , but another error occurs on work node install. here is the cilium pod log.

level=info msg="Auto-disabling \"enable-bpf-clock-probe\" feature since KERNEL_HZ cannot be determined" error="Cannot probe CONFIG_HZ" subsys=daemon
level=info msg="Using autogenerated IPv4 allocation range" subsys=node v4Prefix=10.28.0.0/16
level=info msg="Initializing daemon" subsys=daemon
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=info msg="Establishing connection to apiserver" host="https://127.0.0.1:6443" subsys=k8s
level=error msg="Unable to contact k8s api-server" error="Get \"https://127.0.0.1:6443/api/v1/namespaces/kube-system\": dial tcp 127.0.0.1:6443: connect: connection refused" ipAddr="https://127.0.0.1:6443" subsys=k8s
level=fatal msg="Unable to initialize Kubernetes subsystem" error="unable to create k8s client: unable to create k8s client: Get \"https://127.0.0.1:6443/api/v1/namespaces/kube-system\": dial tcp 127.0.0.1:6443: connect: connection refused" subsys=daemon

It seems the the k8s api url is untouchabled..

snakeliwei commented 1 year ago

The work node works with follow install cmd.

cilium install --helm-set-string=ipam.operator.clusterPoolIPv4PodCIDR="10.42.0.0/16",kubeProxyReplacement=strict,k8sServiceHost=10.0.9.44,k8sServicePort=6443

It seems we also need input the real k8s-api address for cilium install.

xiaods commented 1 year ago

Thanks your report, k8sServiceHost is bad idea for settting, it will add more effort on our deploymnet. i will dig some way to overcome this settings pain.

xiaods commented 1 year ago

@snakeliwei with many case with cilium kubeProxyReplacement=strict I have add env API_SERVER_IP cilium cni插件需要指定api-server的IP, 必须是主机IP to install.sh

Now k8e fully support cilium ctl 1.12