search

projectcalico / calico

Cloud native networking and network security
https://docs.tigera.io/calico/latest/about/
Apache License 2.0
5.7k stars 1.27k forks source link

Calico kube controller pod failling on rhel 9 , but working on rhel 7 #8930

Open saku3071 opened 2 weeks ago

saku3071 commented 2 weeks ago

Calico kube controller pod failling on rhel 9 , but working on rhel 7

version 3.26.1

we moving from cis aws ami rhel 7 to cis aws ami rhel 9 with same eks bootstrap code , however calico kube controllers pods were failing on rhel 9 only

ens-addon and core dns addon were failing too due to this .

pod on eks logs:

2024-06-19 12:58:09.101 [INFO][1] main.go 107: Loaded configuration from environment config=&config.Config{LogLevel:"info", WorkloadEndpointWorkers:1, ProfileWorkers:1, PolicyWorkers:1, NodeWorkers:1, Kubeconfig:"", DatastoreType:"kubernetes"}
W0619 12:58:09.102541       1 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
2024-06-19 12:58:09.102 [INFO][1] main.go 131: Ensuring Calico datastore is initialized
2024-06-19 12:58:39.132 [ERROR][1] client.go 295: Error getting cluster information config ClusterInformation="default" error=Get "https://172.20.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 172.20.0.1:443: i/o timeout
2024-06-19 12:58:39.133 [INFO][1] main.go 138: Failed to initialize datastore error=Get "https://172.20.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 172.20.0.1:443: i/o timeout
2024-06-19 12:59:09.121 [ERROR][1] client.go 295: Error getting cluster information config ClusterInformation="default" error=Get "https://172.20.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default": context deadline exceeded
2024-06-19 12:59:09.121 [INFO][1] main.go 138: Failed to initialize datastore error=Get "https://172.20.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default": context deadline exceeded
2024-06-19 12:59:09.122 [FATAL][1] main.go 151: Failed to initialize Calico datastore

rhel 9 iptables version 1.8 legacy

Does calico works on rhel9 ? any specfiic iptables rules or issues ? or it works on specific iptables version only like 1.46 in rhel 7 ?

saku3071 commented 1 week ago

We are on latest calico 3.28 version and still facing same issue.

caseydavenport commented 4 days ago

I'm not aware of any issues with RHEL or the environment you describe.

The error you're seeing is Calico being unable to communicate with the kube-apiserver (I think via a Service IP) - potentially an issue with the node's underlying network access to the API server, or with kube-proxy service rules.