gardener / etcd-backup-restore

Collection of components to backup and restore the etcd of a Kubernetes cluster.
Apache License 2.0
287 stars 100 forks source link

So many `rejected connection error` in ETCD logs 🔍 #585

Open ishan16696 opened 1 year ago

ishan16696 commented 1 year ago

Describe the bug: There are lot of logs of rejected connection in etcd but clusters(kube-apiserver) is running fine without any issues, I haven't noticed/experienced any problems so far. Logs:

2023-02-06 05:58:46 {"log":"rejected connection from \"127.0.0.1:50234\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
2023-02-06 05:56:56 {"log":"rejected connection from \"127.0.0.1:60860\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
.
.
2023-02-06 05:53:56 {"log":"rejected connection from \"127.0.0.1:42208\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
2023-02-06 05:53:46 {"log":"rejected connection from \"127.0.0.1:49266\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
2023-02-06 05:52:20 {"log":"rejected connection from \"127.0.0.1:33918\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
.
.
2023-02-06 05:48:45 {"log":"rejected connection from \"127.0.0.1:46070\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
.
.
2023-02-06 05:40:49 {"log":"rejected connection from \"127.0.0.1:46676\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
.
.
2023-02-06 05:39:09 {"log":"rejected connection from \"127.0.0.1:53230\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
2023-02-06 05:36:23 {"log":"rejected connection from \"127.0.0.1:52298\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}
2023-02-06 05:35:58 {"log":"rejected connection from \"127.0.0.1:37942\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}

Expected behavior: These logs of connection rejected in ETCD logs shouldn’t be there, so that it is easy to debug any etcd related issue as well as it shouldn’t give DoD a false information.

Environment (please complete the following information):

Anything else we need to know?:

ishan16696 commented 1 year ago

From initial look, it seems some other client requests are coming to etcd and TLS handshake failed which leads to error:

2023-02-06 05:58:46 {"log":"rejected connection from \"127.0.0.1:50234\" (error \"EOF\", ServerName \"etcd-main-local\")","severity":"INFO","source":"embed"}

I will debug it further. /assign