search

bitnami / charts

Bitnami Helm Charts
https://bitnami.com
Other
9k stars 9.22k forks source link

[bitnami/etcd] etcdserver: publish error: etcdserver: request timed out #5953

Closed jicki closed 3 years ago

jicki commented 3 years ago

Which chart: The name (and version) of the affected chart

etcd-6.1.3

Describe the bug A clear and concise description of what the bug is.

kubectl -n dtu-etcd get pods 
NAME     READY   STATUS    RESTARTS   AGE
etcd-0   1/1     Running   0          22m
etcd-1   0/1     Running   4          13m
etcd-2   1/1     Running   0          22m
etcd 07:13:08.91 
etcd 07:13:08.92 Welcome to the Bitnami etcd container
etcd 07:13:08.92 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-etcd
etcd 07:13:08.92 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-etcd/issues
etcd 07:13:08.92 
etcd 07:13:08.93 INFO  ==> ** Starting etcd setup **
etcd 07:13:08.95 INFO  ==> Validating settings in ETCD_* env vars..
etcd 07:13:08.95 WARN  ==> You set the environment variable ALLOW_NONE_AUTHENTICATION=yes. For safety reasons, do not use this flag in a production environment.
etcd 07:13:08.96 INFO  ==> Initializing etcd
etcd 07:13:08.97 INFO  ==> Detected data from previous deployments
etcd 07:13:29.19 WARN  ==> Cluster not responding!
etcd 07:13:29.20 WARN  ==> Disaster recovery is disabled, the cluster will try to recover on it's own

etcd 07:13:29.21 INFO  ==> ** etcd setup finished! **
etcd 07:13:29.24 INFO  ==> ** Starting etcd **
2021-03-30 07:13:29.280528 I | pkg/flags: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=https://etcd-1.etcd-headless.dtu-etcd.svc.cluster.local:2379
2021-03-30 07:13:29.280648 I | pkg/flags: recognized and used environment variable ETCD_CERT_FILE=/opt/bitnami/etcd/certs/etcd-client.pem
2021-03-30 07:13:29.280661 I | pkg/flags: recognized and used environment variable ETCD_CLIENT_CERT_AUTH=true
2021-03-30 07:13:29.280678 I | pkg/flags: recognized and used environment variable ETCD_DATA_DIR=/bitnami/etcd/data
2021-03-30 07:13:29.280713 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://etcd-1.etcd-headless.dtu-etcd.svc.cluster.local:2380
2021-03-30 07:13:29.280735 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER=etcd-0=http://etcd-0.etcd-headless.dtu-etcd.svc.cluster.local:2380,etcd-1=http://etcd-1.etcd-headless.dtu-etcd.svc.cluster.local:2380,etcd-2=http://etcd-2.etcd-headless.dtu-etcd.svc.cluster.local:2380
2021-03-30 07:13:29.280951 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_STATE=new
2021-03-30 07:13:29.280969 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
2021-03-30 07:13:29.280997 I | pkg/flags: recognized and used environment variable ETCD_KEY_FILE=/opt/bitnami/etcd/certs/etcd-client-key.pem
2021-03-30 07:13:29.281012 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379
2021-03-30 07:13:29.281025 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
2021-03-30 07:13:29.281033 I | pkg/flags: recognized and used environment variable ETCD_LOG_LEVEL=info
2021-03-30 07:13:29.281046 I | pkg/flags: recognized and used environment variable ETCD_NAME=etcd-1
2021-03-30 07:13:29.281069 I | pkg/flags: recognized and used environment variable ETCD_PEER_AUTO_TLS=false
2021-03-30 07:13:29.281099 I | pkg/flags: recognized and used environment variable ETCD_TRUSTED_CA_FILE=/opt/bitnami/etcd/certs/ca.pem
2021-03-30 07:13:29.281113 W | pkg/flags: unrecognized environment variable ETCD_EXTERNAL_PORT_2379_TCP_PORT=2379
2021-03-30 07:13:29.281119 W | pkg/flags: unrecognized environment variable ETCD_EXTERNAL_PORT=tcp://172.13.63.18:2379
2021-03-30 07:13:29.281125 W | pkg/flags: unrecognized environment variable ETCD_ON_K8S=yes
2021-03-30 07:13:29.281129 W | pkg/flags: unrecognized environment variable ETCD_SNAPSHOTS_DIR=/snapshots
2021-03-30 07:13:29.281134 W | pkg/flags: unrecognized environment variable ETCD_BIN_DIR=/opt/bitnami/etcd/sbin
2021-03-30 07:13:29.281138 W | pkg/flags: unrecognized environment variable ETCD_EXTERNAL_PORT_2379_TCP_ADDR=172.13.63.18
2021-03-30 07:13:29.281142 W | pkg/flags: unrecognized environment variable ETCD_VOLUME_DIR=/bitnami/etcd
2021-03-30 07:13:29.281147 W | pkg/flags: unrecognized environment variable ETCD_EXTERNAL_PORT_2379_TCP=tcp://172.13.63.18:2379
2021-03-30 07:13:29.281152 W | pkg/flags: unrecognized environment variable ETCD_ROOT_PASSWORD=
2021-03-30 07:13:29.281157 W | pkg/flags: unrecognized environment variable ETCD_CLUSTER_DOMAIN=etcd-headless.dtu-etcd.svc.cluster.local
2021-03-30 07:13:29.281161 W | pkg/flags: unrecognized environment variable ETCD_DISASTER_RECOVERY=no
2021-03-30 07:13:29.281166 W | pkg/flags: unrecognized environment variable ETCD_EXTERNAL_SERVICE_HOST=172.13.63.18
2021-03-30 07:13:29.281170 W | pkg/flags: unrecognized environment variable ETCD_EXTERNAL_SERVICE_PORT_ETCD=2379
2021-03-30 07:13:29.281174 W | pkg/flags: unrecognized environment variable ETCD_DAEMON_GROUP=etcd
2021-03-30 07:13:29.281179 W | pkg/flags: unrecognized environment variable ETCD_EXTERNAL_PORT_2379_TCP_PROTO=tcp
2021-03-30 07:13:29.281183 W | pkg/flags: unrecognized environment variable ETCD_START_FROM_SNAPSHOT=no
2021-03-30 07:13:29.281189 W | pkg/flags: unrecognized environment variable ETCD_INIT_SNAPSHOTS_DIR=/init-snapshot
2021-03-30 07:13:29.281193 W | pkg/flags: unrecognized environment variable ETCD_EXTERNAL_SERVICE_PORT=2379
2021-03-30 07:13:29.281199 W | pkg/flags: unrecognized environment variable ETCD_BASE_DIR=/opt/bitnami/etcd
2021-03-30 07:13:29.281205 W | pkg/flags: unrecognized environment variable ETCD_NEW_MEMBERS_ENV_FILE=/bitnami/etcd/data/new_member_envs
2021-03-30 07:13:29.281209 W | pkg/flags: unrecognized environment variable ETCD_DAEMON_USER=etcd
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2021-03-30 07:13:29.281268 I | etcdmain: etcd Version: 3.4.15
2021-03-30 07:13:29.281277 I | etcdmain: Git SHA: aa7126864
2021-03-30 07:13:29.281282 I | etcdmain: Go Version: go1.12.17
2021-03-30 07:13:29.281287 I | etcdmain: Go OS/Arch: linux/amd64
2021-03-30 07:13:29.281292 I | etcdmain: setting maximum number of CPUs to 24, total number of available CPUs is 24
2021-03-30 07:13:29.281367 N | etcdmain: the server is already initialized as member before, starting as etcd member...
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2021-03-30 07:13:29.281653 I | embed: name = etcd-1
2021-03-30 07:13:29.281668 I | embed: data dir = /bitnami/etcd/data
2021-03-30 07:13:29.281673 I | embed: member dir = /bitnami/etcd/data/member
2021-03-30 07:13:29.281678 I | embed: heartbeat = 100ms
2021-03-30 07:13:29.281682 I | embed: election = 1000ms
2021-03-30 07:13:29.281686 I | embed: snapshot count = 100000
2021-03-30 07:13:29.281695 I | embed: advertise client URLs = https://etcd-1.etcd-headless.dtu-etcd.svc.cluster.local:2379
2021-03-30 07:13:29.281701 I | embed: initial advertise peer URLs = http://etcd-1.etcd-headless.dtu-etcd.svc.cluster.local:2380
2021-03-30 07:13:29.281707 I | embed: initial cluster = 
2021-03-30 07:13:29.283630 I | etcdserver: restarting member 59e70374362c213c in cluster ea14cda42055c997 at commit index 0
raft2021/03/30 07:13:29 INFO: 59e70374362c213c switched to configuration voters=()
raft2021/03/30 07:13:29 INFO: 59e70374362c213c became follower at term 0
raft2021/03/30 07:13:29 INFO: newRaft 59e70374362c213c [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2021-03-30 07:13:29.287166 W | auth: simple token is not cryptographically signed
2021-03-30 07:13:29.290804 I | etcdserver: starting server... [version: 3.4.15, cluster version: to_be_decided]
2021-03-30 07:13:29.295641 I | embed: ClientTLS: cert = /opt/bitnami/etcd/certs/etcd-client.pem, key = /opt/bitnami/etcd/certs/etcd-client-key.pem, trusted-ca = /opt/bitnami/etcd/certs/ca.pem, client-cert-auth = true, crl-file = 
2021-03-30 07:13:29.295817 I | embed: listening for peers on [::]:2380
2021-03-30 07:13:36.292552 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:13:43.293161 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:13:50.293681 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:13:57.294022 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:14:04.294514 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:14:11.294927 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:14:18.295298 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:14:25.295601 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:14:32.296080 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:14:39.297026 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:14:46.297276 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:14:53.297734 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:00.298365 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:07.298672 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:14.299177 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:21.299785 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:28.300198 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:35.300589 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:42.300986 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:49.301389 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:15:56.301828 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:16:03.302316 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:16:10.302728 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:16:17.303175 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:16:24.303939 E | etcdserver: publish error: etcdserver: request timed out
2021-03-30 07:16:31.304331 E | etcdserver: publish error: etcdserver: request timed out
================> 获取集群状态
+------------------+---------+--------+-------------------------------------------------------------+--------------------------------------------------------------+------------+
|        ID        | STATUS  |  NAME  |                         PEER ADDRS                          |                         CLIENT ADDRS                         | IS LEARNER |
+------------------+---------+--------+-------------------------------------------------------------+--------------------------------------------------------------+------------+
|  cc8f63b712650a8 | started | etcd-0 | http://etcd-0.etcd-headless.dtu-etcd.svc.cluster.local:2380 | https://etcd-0.etcd-headless.dtu-etcd.svc.cluster.local:2379 |      false |
| 6521db4a51c6590c | started | etcd-2 | http://etcd-2.etcd-headless.dtu-etcd.svc.cluster.local:2380 | https://etcd-2.etcd-headless.dtu-etcd.svc.cluster.local:2379 |      false |
+------------------+---------+--------+-------------------------------------------------------------+--------------------------------------------------------------+------------+
+-------------------------+-----------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|        ENDPOINT         |       ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+-------------------------+-----------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://10.9.9.123:2379 | cc8f63b712650a8 |  3.4.15 |  721 kB |     false |      false |        13 |      10011 |              10011 |        |
+-------------------------+-----------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
miguelaeh commented 3 years ago

Hi @jicki , Could you please provide more information? How are you installing the chart, what are the steps to reproduce the issue, etc

github-actions[bot] commented 3 years ago

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

github-actions[bot] commented 3 years ago

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.