search

cloudnativebooks / cloud-native-istio

华为云原生丛书之《云原生服务网格Istio：原理、实践、架构与源码解析》
Apache License 2.0
341 stars 143 forks source link

部署weather-gateway的时候错误 #23

Closed xxlaila closed 4 years ago

xxlaila commented 5 years ago

istio 我参考官方使用helm进行安装，版本是1.2.8的。 参考第8章部署weather的时候没问题，在部署weather-gateway的时候提示错误

# kubectl apply -f install/weather-gateway.yaml 
Error from server (Timeout): error when creating "install/weather-gateway.yaml": Timeout: request did not complete within requested timeout 30s
Error from server (Timeout): error when creating "install/weather-gateway.yaml": Timeout: request did not complete within requested timeout 30s
Error from server (Timeout): error when creating "install/weather-gateway.yaml": Timeout: request did not complete within requested timeout 30s

在使用istio官方的例子来部署gateway的时候也是提示这个，搞不明白这是什么问题，k8s是1.14.6的，在自己本地使用虚拟机用二进制方式来进行部署的。 这是我安装istio的时候使用的参数，

helm install install/kubernetes/helm/istio --wait \
    --name istio \
    --namespace istio-system \
    --set global.mtls.enabled=true \
    --set kiali.enabled=true \
    --set tracing.enabled=true \
    --set grafana.enabled=true \
    --set servicegraph.enabled=true \
    --set global.k8sIngress.enabled=true \
    --set global.k8sIngress.gatewayName=ingressgateway \
    --set kiali.createDemoSecret=true \
    --set kiali.contextPath=/ \
    --set "kiali.dashboard.jaegerURL=http://jaeger-query:16686" \
    --set "kiali.dashboard.grafanaURL=http://grafana:3000" \
    --set gateways.istio-ingressgateway.type=NodePort \
    --set gateways.istio-egressgateway.type=NodePort \
    --set sidecarInjectorWebhook.enabled=true
hzxuzhonghu commented 5 years ago

排查一下k8s api server访问问题，看起来是访问超时，你可以进一步看看apiserver的日志。

xxlaila commented 5 years ago

我查看了apiserver 的日志，

Nov 08 09:59:33 k8s-master-01-3.kxl kube-apiserver[31393]: I1108 09:59:33.659161   31393 trace.go:81] Trace[40457478]: "Create /apis/networking.istio.io/v1alpha3/namespaces/istio-system/gateways" (started: 2019-11-08 09:59:03.657132211 +0800 CST m=+328870.679516549) (total time: 30.001964129s):
Nov 08 09:59:33 k8s-master-01-3.kxl kube-apiserver[31393]: Trace[40457478]: [30.001964129s] [30.001043358s] END
Nov 08 09:59:33 k8s-master-01-3.kxl kube-apiserver[31393]: W1108 09:59:33.659790   31393 dispatcher.go:73] Failed calling webhook, failing closed pilot.validation.istio.io: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Nov 08 09:59:39 k8s-master-01-3.kxl kube-apiserver[31393]: I1108 09:59:39.979543   31393 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
Nov 08 10:00:03 k8s-master-01-3.kxl kube-apiserver[31393]: W1108 10:00:03.764977   31393 dispatcher.go:73] Failed calling webhook, failing closed pilot.validation.istio.io: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Nov 08 10:00:03 k8s-master-01-3.kxl kube-apiserver[31393]: I1108 10:00:03.765401   31393 trace.go:81] Trace[1649710078]: "Create /apis/networking.istio.io/v1alpha3/namespaces/default/destinationrules" (started: 2019-11-08 09:59:33.763211641 +0800 CST m=+328900.785596022) (total time: 30.00209862s):
Nov 08 10:00:03 k8s-master-01-3.kxl kube-apiserver[31393]: Trace[1649710078]: [30.00209862s] [30.001534667s] END
Nov 08 10:00:33 k8s-master-01-3.kxl kube-apiserver[31393]: I1108 10:00:33.840606   31393 trace.go:81] Trace[970347589]: "Create /apis/networking.istio.io/v1alpha3/namespaces/weather/virtualservices" (started: 2019-11-08 10:00:03.83792882 +0800 CST m=+328930.860313362) (total time: 30.002612137s):
Nov 08 10:00:33 k8s-master-01-3.kxl kube-apiserver[31393]: Trace[970347589]: [30.002612137s] [30.001075132s] END
Nov 08 10:00:33 k8s-master-01-3.kxl kube-apiserver[31393]: W1108 10:00:33.841663   31393 dispatcher.go:73] Failed calling webhook, failing closed pilot.validation.istio.io: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Nov 08 10:00:38 k8s-master-01-3.kxl kube-apiserver[31393]: I1108 10:00:38.260710   31393 trace.go:81] Trace[460935607]: "GuaranteedUpdate etcd3: *v1.Endpoints" (started: 2019-11-08 10:00:37.644096515 +0800 CST m=+328964.666480867) (total time: 616.515599ms):
Nov 08 10:00:38 k8s-master-01-3.kxl kube-apiserver[31393]: Trace[460935607]: [533.664848ms] [449.34458ms] Transaction prepared
Nov 08 10:00:39 k8s-master-01-3.kxl kube-apiserver[31393]: I1108 10:00:39.986622   31393 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
Nov 08 10:01:38 k8s-master-01-3.kxl kube-apiserver[31393]: I1108 10:01:38.780611   31393 trace.go:81] Trace[269873276]: "Get /api/v1/namespaces/default" (started: 2019-11-08 10:01:37.631910347 +0800 CST m=+329024.654294682) (total time: 1.148554735s):
Nov 08 10:01:38 k8s-master-01-3.kxl kube-apiserver[31393]: Trace[269873276]: [1.148211464s] [1.148180236s] About to write a response
xxlaila commented 5 years ago

查看 istio-pilot日志发现

# kubectl logs istio-pilot-569499d666-rfjsh  -n istio-system discovery
2019-11-08T07:26:14.097765Z info    Handling event update for pod istio-security-post-install-1.2.8-c52np in namespace istio-system -> 172.30.112.9
2019-11-08T07:26:27.395268Z info    Handling event update for pod istio-security-post-install-1.2.8-c52np in namespace istio-system -> 172.30.112.9
2019-11-08T07:26:38.227484Z info    Client received GoAway with http2.ErrCodeEnhanceYourCalm.
2019-11-08T07:26:38.227760Z info    pickfirstBalancer: HandleSubConnStateChange: 0xc0001fbaa0, CONNECTING
2019-11-08T07:26:38.228913Z info    transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2019-11-08T07:26:38.230352Z error   mcp Error receiving MCP resource: rpc error: code = Unavailable desc = transport is closing
2019-11-08T07:26:38.230387Z error   mcp Error receiving MCP response: rpc error: code = Unavailable desc = transport is closing
2019-11-08T07:26:38.235755Z info    pickfirstBalancer: HandleSubConnStateChange: 0xc0001fbaa0, READY
2019-11-08T07:26:39.230701Z info    mcp (re)trying to establish new MCP sink stream

istio-galley日志

# kubectl logs istio-galley-64f7d8cc97-8nbpc  -n istio-system
2019-11-08T07:23:38.860184Z info    mcp MCP: connection {addr=172.30.104.7:57190 id=3} ACK collection=istio/rbac/v1alpha1/serviceroles with version="0" nonce="16" inc=false
2019-11-08T07:23:38.860197Z info    mcp Watch(): created watch 28 for istio/rbac/v1alpha1/serviceroles from group "default", version "0"
2019-11-08T07:23:38.860217Z info    mcp MCP: connection {addr=172.30.104.7:57190 id=3} ACK collection=istio/networking/v1alpha3/gateways with version="0" nonce="17" inc=false
2019-11-08T07:23:38.860268Z info    mcp Watch(): created watch 29 for istio/networking/v1alpha3/gateways from group "default", version "0"
2019-11-08T07:26:38.227268Z info    transport: Got too many pings from the client, closing the connection.
2019-11-08T07:26:38.227414Z info    transport: loopyWriter.run returning. Err: transport: Connection closing
2019-11-08T07:26:38.228857Z info    transport: http2Server.HandleStreams failed to read frame: read tcp 172.30.104.4:9901->172.30.104.7:57190: use of closed network connection
2019-11-08T07:26:38.229130Z error   mcp MCP: connection {addr=172.30.104.7:57190 id=3}: TERMINATED with errors: rpc error: code = Canceled desc = context canceled
2019-11-08T07:26:38.229162Z info    mcp MCP: connection {addr=172.30.104.7:57190 id=3}: CLOSED
2019-11-08T07:26:39.232906Z info    mcp MCP: connection {addr=172.30.104.7:57502 id=4}: NEW (ResourceSource), supported collections: []string{"istio/config/v1alpha2/legacy/checknothings", "istio/policy/v1beta1/handlers", "istio/config/v1alpha2/httpapispecbindings", "istio/networking/v1alpha3/serviceentries", "istio/config/v1alpha2/legacy/authorizations", "istio/rbac/v1alpha1/serviceroles", "istio/config/v1alpha2/httpapispecs", "istio/config/v1alpha2/legacy/stackdrivers", "istio/config/v1alpha2/legacy/metrics", "istio/policy/v1beta1/rules", "istio/config/v1alpha2/legacy/reportnothings", "istio/config/v1alpha2/legacy/signalfxs", "istio/config/v1alpha2/legacy/tracespans", "istio/config/v1alpha2/legacy/apikeys", "istio/config/v1alpha2/templates", "istio/config/v1alpha2/legacy/noops", "istio/config/v1alpha2/adapters", "istio/networking/v1alpha3/sidecars", "istio/rbac/v1alpha1/clusterrbacconfigs", "istio/config/v1alpha2/legacy/bypasses", "istio/config/v1alpha2/legacy/opas", "istio/mesh/v1alpha1/MeshConfig", "istio/config/v1alpha2/legacy/redisquotas", "istio/networking/v1alpha3/gateways", "istio/networking/v1alpha3/envoyfilters", "istio/authentication/v1alpha1/meshpolicies", "istio/config/v1alpha2/legacy/listentries", "istio/config/v1alpha2/legacy/stdios", "istio/config/v1alpha2/legacy/cloudwatches", "istio/config/v1alpha2/legacy/kuberneteses", "istio/config/v1alpha2/legacy/deniers", "istio/config/v1alpha2/legacy/prometheuses", "istio/config/v1alpha2/legacy/edges", "istio/mixer/v1/config/client/quotaspecs", "istio/config/v1alpha2/legacy/memquotas", "istio/config/v1alpha2/legacy/rbacs", "istio/networking/v1alpha3/synthetic/serviceentries", "istio/authentication/v1alpha1/policies", "k8s/extensions/v1beta1/ingresses", "istio/networking/v1alpha3/destinationrules", "istio/mixer/v1/config/client/quotaspecbindings", "istio/rbac/v1alpha1/rbacconfigs", "istio/policy/v1beta1/attributemanifests", "istio/config/v1alpha2/legacy/circonuses", "istio/config/v1alpha2/legacy/fluentds", "istio/config/v1alpha2/legacy/zipkins", "istio/config/v1alpha2/legacy/listcheckers", "istio/config/v1alpha2/legacy/logentries", "istio/config/v1alpha2/legacy/quotas", "istio/rbac/v1alpha1/authorizationpolicies", "istio/networking/v1alpha3/virtualservices", "istio/rbac/v1alpha1/servicerolebindings", "istio/config/v1alpha2/legacy/statsds", "istio/config/v1alpha2/legacy/kubernetesenvs", "istio/config/v1alpha2/legacy/dogstatsds", "istio/config/v1alpha2/legacy/solarwindses", "istio/policy/v1beta1/instances"}
xxlaila commented 5 years ago

我已经解决了，谢谢🙏指点