search

apache / apisix

The Cloud-Native API Gateway
https://apisix.apache.org/blog/
Apache License 2.0
14.3k stars 2.49k forks source link

bug: Error: etcdserver: member not found #11386

Open sudhirs649 opened 2 months ago

sudhirs649 commented 2 months ago

Current Behavior

We are using APISIX incorporated ETCD, which is present APISIX helmet chart https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix/charts/etcd-9.7.3.tgz.

So it is only a part of APISIX. Please at least guide us on this issue.

We are getting this error even though we upgraded the etcd 8.7 to etcd 9.7 but still facing the same error. Our apisix pod image tag: 3.8.0-debian

Expected Behavior

all apsisix-etcd pod needs to be in running status

Error Logs

etcd 05:04:16.59 etcd 05:04:16.74 Welcome to the Bitnami etcd container etcd 05:04:16.75 Subscribe to project updates by watching https://github.com/bitnami/containers etcd 05:04:16.77 Submit issues and feature requests at https://github.com/bitnami/containers/issues etcd 05:04:16.78 etcd 05:04:16.79 INFO ==> Starting etcd setup etcd 05:04:17.35 INFO ==> Validating settings in ETCD_* env vars.. etcd 05:04:17.37 WARN ==> You set the environment variable ALLOW_NONE_AUTHENTICATION=yes. For safety reasons, do not use this flag in a production environment. etcd 05:04:17.55 INFO ==> Initializing etcd etcd 05:04:17.66 INFO ==> Generating etcd config file using env variables etcd 05:04:18.47 INFO ==> Detected data from previous deployments etcd 05:04:20.50 INFO ==> Updating member in existing cluster {"level":"warn","ts":"2024-07-02T05:04:22.015Z","logger":"etcd-client","caller":"v3@v3.5.7/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00055c000/accel-apisix-etcd-0.accel-apisix-etcd-headless.accel-apisix.svc.cluster.local:2379","attempt":0,"error":"rpc error: code = NotFound desc = etcdserver: member not found"} Error: etcdserver: member not found

Steps to Reproduce

We have deployed through helm charts of apisix we are only using etcd as a dependency chart

https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix/charts/etcd-9.7.3.tgz.

Environment

APISIX version (run apisix version)--> 3.8.0-debian
ETCD package version --> etcd-9.7.3.tgz , image version docker.io/bitnami/etcd:3.5.7-debian-11-r14
Operating system (run uname -a): linux , Kernel version 5.15.0-1061-azure
OpenResty / Nginx version (run openresty -V or nginx -V): helm.sh/chart:ingress-nginx-4.10.0
APISIX Dashboard version, if relevant: 3.0.0-alpine
AKS cluster version - 1.28
derekkl commented 1 month ago

We are having this issue as well.