Open jishaashokan opened 1 year ago
jishaashokan
commented
1 year ago /ingress-apisix # ./apisix-ingress-controller version --long Version: 1.6.0 Git SHA: no-git-module Go Version: go1.19.4 Building OS/Arch: linux/amd64 Running OS/Arch: linux/amd64
Kubernetes version 1.22
jishaashokan
commented
1 year ago This example fails : https://apisix.apache.org/docs/ingress-controller/tutorials/proxy-grpc-service/
kubectl run -it -n ingress-apisix --rm grpcurl --restart=Never --image=quay.io/mhausenblas/gump:0.1 -- sh If you don't see a command prompt, try pressing enter. /go $ grpcurl --plaintext yages:9000 yages.Echo.Ping { "text": "pong" } /go $ grpcurl --insecure -servername grpc-proxy apisix-gateway:443 yages.Echo.Ping Error invoking method "yages.Echo.Ping": failed to query for service descriptor "yages.Echo": rpc error: code = Internal desc = transport: received the unexpected content-type "text/plain; charset=utf-8" /go $ exit
`mk describe au yages
Name: yages
Namespace: ingress-apisix
Labels:
Normal ResourcesSynced 2m31s (x6 over 24m) ApisixIngress ApisixIngress synced successfully `
`mk describe ar grpc-proxy-route
Name: grpc-proxy-route
Namespace: ingress-apisix
Labels:
Normal ResourcesSynced 69s (x7 over 23m) ApisixIngress ApisixIngress synced successfully `
`mk describe apisixtls.apisix.apache.org/grpc-secret
Name: grpc-secret
Namespace: ingress-apisix
Labels:
Normal ResourcesSynced 4m32s (x14 over 61m) ApisixIngress ApisixIngress synced successfully `
tao12345666333
commented
1 year ago apisix-etcd-1 0/1 CrashLoopBackOff 21 (3m45s ago) 115myou can check the pod's log
Issue description
apisix was installed using :
git clone https://github.com/apache/apisix-helm-chart.git helm repo add bitnami https://charts.bitnami.com/bitnami helm dependency update ./charts/apisix helm install apisix apisix/apisix \ --set gateway.type=LoadBalancer \ --set allow.ipList="{0.0.0.0/0}" \ --set ingress-controller.enabled=true \ --namespace ingress-apisix \ --set ingress-controller.config.apisix.serviceNamespace=ingress-apisix \ --set gateway.tls.enabled=true \ --set ingress-controller.config.apisix.adminKey=xxxx \ --set admin.credentials.admin=xxxx \ --set admin.credentials.viewer=yyyy \ --set ingressController.config.apisix.baseURL=http://apisix-admin:9180/apisix/admin
The following pods come up, but apisix-etcd-1 crashes from the beginning.
NAME READY STATUS RESTARTS AGE apisix-54cb677c56-n8jjp 1/1 Running 0 178m apisix-dashboard-78dc4dc8bb-h9bnb 1/1 Running 2 (106m ago) 2d apisix-etcd-0 1/1 Running 0 161m apisix-etcd-1 0/1 CrashLoopBackOff 21 (3m45s ago) 115m apisix-etcd-2 1/1 Running 0 70m
kubectl exec -it apisix-etcd-2 -- etcdctl endpoint health 127.0.0.1:2379 is healthy: successfully committed proposal: took = 4.01932ms
kubectl exec -it apisix-etcd-1 -- etcdctl endpoint health {"level":"warn","ts":"2022-12-28T07:49:45.052Z","logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0002e6a80/127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"} 127.0.0.1:2379 is unhealthy: failed to commit proposal: context deadline exceeded Error: unhealthy cluster command terminated with exit code 1
kubectl exec -it apisix-etcd-0 -- etcdctl endpoint health 127.0.0.1:2379 is healthy: successfully committed proposal: took = 2.962883ms
Also, after installing Apisix Route, Upstream and TLS, the apisix-ingress-controller logs gives the following errors :
kubectl logs -f apisix-ingress-controller-6c4fbd9697-q9whw
2022-12-28T13:30:17+08:00 error apisix/upstream.go:75 failed to get upstream from APISIX {"name": "ingress-apisix_apisix-etcd_2379", "url": "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/29b61b45", "cluster": "default", "error": "Get \"http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/29b61b45\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"} 2022-12-28T13:30:17+08:00 error types/types.go:218 failed to get upstream {"upstream": "ingress-apisix_apisix-etcd_2379", "cluster": "name=default; base_url=http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin", "error": "Get \"http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/29b61b45\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"} 2022-12-28T13:30:17+08:00 warn endpoint/endpoint.go:130 sync endpoints failed, will retry {"object": {"Type":2,"Object":{},"OldObject":null,"Tombstone":null}} 2022-12-28T13:30:17+08:00 error apisix/route.go:163 failed to create route: Put "http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/routes/9a4e702d": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2022-12-28T13:30:17+08:00 warn apisix/apisix_route.go:513 sync ApisixRoute failed, will retry {"object": {"Type":1,"Object":{"Key":"ingress-apisix/grpc-route","OldObject":null,"GroupVersion":"apisix.apache.org/v2"},"OldObject":null,"Tombstone":null}, "error": "2 errors occurred:\n\t Put \"http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/upstreams/69084189\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n\t Put \"http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/routes/9a4e702d\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n\n"} 2022-12-28T13:30:18+08:00 error apisix/apisix_tls.go:213 failed to sync SSL to APISIX {"error": "Put \"http://apisix-admin.ingress-apisix.svc.cluster.local:9180/apisix/admin/ssls/46a9cf0b\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)", "ssl": {"id":"46a9cf0b","snis":["grpc-proxy"],"cert":"
Environment