search

hystax / optscale

FinOps, MLOps and cloud cost optimization tool. Supports AWS, Azure, GCP, Alibaba Cloud and Kubernetes.
https://hystax.com
Apache License 2.0
1.18k stars 165 forks source link

All certificates invalid #236

Closed tguisep closed 5 months ago

tguisep commented 5 months ago

Hello, I had issue with my optscale instance ... all certificates have expired resulting a general unavailability.

Some Logs:

I0318 12:44:17.649194       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379 0  <nil>}]
W0318 12:44:17.651382       1 clientconn.go:1120] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 0  <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid". Reconnecting...
I0318 12:44:18.646737       1 client.go:361] parsed scheme: "endpoint"

2024-03-18 12:54:08.807035 I | embed: rejected connection from "127.0.0.1:56140" (error "remote error: tls: bad certificate", ServerName "")
2024-03-18 12:54:25.584942 I | embed: rejected connection from "127.0.0.1:37478" (error "tls: failed to verify client's certificate: x509: certificate has expired or is not yet valid", ServerName "")

ubuntu@ip-10-130-1-3:~$ sudo kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[check-expiration] Error reading configuration from the Cluster. Falling back to default configuration

W0318 12:56:29.577489  360086 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0318 12:56:29.577514  360086 validation.go:28] Cannot validate kubelet config - no validator is available
CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Mar 15, 2024 13:28 UTC   <invalid>                               no      
apiserver                  Mar 15, 2024 13:28 UTC   <invalid>       ca                      no      
apiserver-etcd-client      Mar 15, 2024 13:28 UTC   <invalid>       etcd-ca                 no      
apiserver-kubelet-client   Mar 15, 2024 13:28 UTC   <invalid>       ca                      no      
controller-manager.conf    Mar 15, 2024 13:28 UTC   <invalid>                               no      
etcd-healthcheck-client    Mar 15, 2024 13:28 UTC   <invalid>       etcd-ca                 no      
etcd-peer                  Mar 15, 2024 13:28 UTC   <invalid>       etcd-ca                 no      
etcd-server                Mar 18, 2025 12:54 UTC   <invalid>            etcd-ca                 no      
front-proxy-client         Mar 15, 2024 13:28 UTC   <invalid>       front-proxy-ca          no      
scheduler.conf             Mar 15, 2024 13:28 UTC   <invalid>                               no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Mar 13, 2033 13:28 UTC   8y              no      
etcd-ca                 Mar 13, 2033 13:28 UTC   8y              no      
front-proxy-ca          Mar 13, 2033 13:28 UTC   8y              no

Fix: sudo kubeadm alpha certs renew <cert_name>

I'm not running on the last version, idk if a path has been done recently on it.

Thomas.

maxb-hystax commented 5 months ago

Hi! Thank you for the finding! The following command can be used to renew all k8s certificates: kubeadm alpha certs renew all and then restart kubelet service.