search

istio / istio

Connect, secure, control, and observe services.
https://istio.io
Apache License 2.0
35.68k stars 7.68k forks source link

Install on virtual cluster(nested) #35431

Closed ju187 closed 2 years ago

ju187 commented 2 years ago

Bug Description

When doing a POC for nested clusters, the istio pod was stuck in pending status and a describe show no events. The two flavor I tried are vcluster and cluster api provider nested. First of all, does istio support nested clusters?

Version

istio: 1.11.3
KInd: 0.11

Additional Information

No response

howardjohn commented 2 years ago

We will need more info, such as the pod yaml and describe output to make any progress on this

ju187 commented 2 years ago

Hi: The output of describe pod has been attached.  It was for a vcluster created under a 1.18.20 eks cluster.  Thanks. Tony

On Friday, October 1, 2021, 08:53:31 AM PDT, John Howard ***@***.***> wrote:

We will need more info, such as the pod yaml and describe output to make any progress on this

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

howardjohn commented 2 years ago

I don't think email attachments work on github.

ju187 commented 2 years ago

Here it is Name:           istio-ingressgateway-565d647f8c-cxwfx Namespace:      istio-system Priority:       0 Node:           Labels:         app=istio-ingressgateway                 chart=gateways                 heritage=Tiller                 install.operator.istio.io/owning-resource=unknown                 istio=ingressgateway                 istio.io/rev=default                 operator.istio.io/component=IngressGateways                 pod-template-hash=565d647f8c                 release=istio                 service.istio.io/canonical-name=istio-ingressgateway                 service.istio.io/canonical-revision=latest                 sidecar.istio.io/inject=false Annotations:    prometheus.io/path: /stats/prometheus                 prometheus.io/port: 15020                 prometheus.io/scrape: true                 sidecar.istio.io/inject: false Status:         Pending IP:              IPs:            Controlled By:  ReplicaSet/istio-ingressgateway-565d647f8c Containers:   istio-proxy:     Image:       docker.io/istio/proxyv2:1.10.3     Ports:       15021/TCP, 8080/TCP, 8443/TCP, 15090/TCP     Host Ports:  0/TCP, 0/TCP, 0/TCP, 0/TCP     Args:       proxy       router       --domain       $(POD_NAMESPACE).svc.cluster.local       --proxyLogLevel=warning       --proxyComponentLogLevel=misc:error       --log_output_level=default:info       --serviceCluster       istio-ingressgateway     Limits:       cpu:     2       memory:  1Gi     Requests:       cpu:      100m       memory:   128Mi     Readiness:  http-get http://:15021/healthz/ready delay=1s timeout=1s period=2s #success=1 #failure=30     Environment:       JWT_POLICY:                   third-party-jwt       PILOT_CERT_PROVIDER:          istiod       CA_ADDR:                      istiod.istio-system.svc:15012       NODE_NAME:                     (v1:spec.nodeName)       POD_NAME:                     istio-ingressgateway-565d647f8c-cxwfx (v1:metadata.name)       POD_NAMESPACE:                istio-system (v1:metadata.namespace)       INSTANCE_IP:                   (v1:status.podIP)       HOST_IP:                       (v1:status.hostIP)       SERVICE_ACCOUNT:               (v1:spec.serviceAccountName)       CANONICAL_SERVICE:             (v1:metadata.labels['service.istio.io/canonical-name'])       CANONICAL_REVISION:            (v1:metadata.labels['service.istio.io/canonical-revision'])       ISTIO_META_WORKLOAD_NAME:     istio-ingressgateway       ISTIO_META_OWNER:             kubernetes://apis/apps/v1/namespaces/istio-system/deployments/istio-ingressgateway       ISTIO_META_MESH_ID:           cluster.local       TRUST_DOMAIN:                 cluster.local       ISTIO_META_UNPRIVILEGED_POD:  true       ISTIO_META_ROUTER_MODE:       standard       ISTIO_META_CLUSTER_ID:        Kubernetes     Mounts:       /etc/istio/config from config-volume (rw)       /etc/istio/ingressgateway-ca-certs from ingressgateway-ca-certs (ro)       /etc/istio/ingressgateway-certs from ingressgateway-certs (ro)       /etc/istio/pod from podinfo (rw)       /etc/istio/proxy from istio-envoy (rw)       /var/lib/istio/data from istio-data (rw)       /var/run/secrets/istio from istiod-ca-cert (rw)       /var/run/secrets/kubernetes.io/serviceaccount from istio-ingressgateway-service-account-token-wxgjk (ro)       /var/run/secrets/tokens from istio-token (ro) Conditions:   Type           Status   PodScheduled   False Volumes:   istiod-ca-cert:     Type:      ConfigMap (a volume populated by a ConfigMap)     Name:      istio-ca-root-cert     Optional:  false   podinfo:     Type:  DownwardAPI (a volume populated by information about the pod)     Items:       metadata.labels -> labels       metadata.annotations -> annotations       limits.cpu -> cpu-limit       requests.cpu -> cpu-request   istio-envoy:     Type:       EmptyDir (a temporary directory that shares a pod's lifetime)     Medium:          SizeLimit:    istio-data:     Type:       EmptyDir (a temporary directory that shares a pod's lifetime)     Medium:          SizeLimit:    istio-token:     Type:                    Projected (a volume that contains injected data from multiple sources)     TokenExpirationSeconds:  43200   config-volume:     Type:      ConfigMap (a volume populated by a ConfigMap)     Name:      istio     Optional:  true   ingressgateway-certs:     Type:        Secret (a volume populated by a Secret)     SecretName:  istio-ingressgateway-certs     Optional:    true   ingressgateway-ca-certs:     Type:        Secret (a volume populated by a Secret)     SecretName:  istio-ingressgateway-ca-certs     Optional:    true   istio-ingressgateway-service-account-token-wxgjk:     Type:        Secret (a volume populated by a Secret)     SecretName:  istio-ingressgateway-service-account-token-wxgjk     Optional:    false QoS Class:       Burstable Node-Selectors:  Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s                  node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events:   Type     Reason            Age                 From               Message   ----     ------            ----                ----               -------   Warning  FailedScheduling  19s (x22 over 13m)  default-scheduler  0/3 nodes are available: 1 Too many pods, 1 node(s) had taint {sonic-node-group: api}, that the pod didn't tolerate, 1 node(s) had taint {sonic-node-group: nff}, that the pod didn't tolerate.

On Friday, October 1, 2021, 12:56:50 PM PDT, John Howard ***@***.***> wrote:

I don't think email attachments work on github.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

howardjohn commented 2 years ago

sonic-node-group: nff is what is breaking it. This isn't an Istio issue but rather your cluster is not configured to allow your nodes to schedule pods.

On Fri, Oct 1, 2021 at 1:54 PM ju187 @.***> wrote:

Here it is Name: istio-ingressgateway-565d647f8c-cxwfx Namespace: istio-system Priority: 0 Node: Labels: app=istio-ingressgateway chart=gateways heritage=Tiller install.operator.istio.io/owning-resource=unknown istio=ingressgateway istio.io/rev=default operator.istio.io/component=IngressGateways pod-template-hash=565d647f8c release=istio service.istio.io/canonical-name=istio-ingressgateway service.istio.io/canonical-revision=latest sidecar.istio.io/inject=false Annotations: prometheus.io/path: /stats/prometheus prometheus.io/port: 15020 prometheus.io/scrape: true sidecar.istio.io/inject: false Status: Pending IP: IPs: Controlled By: ReplicaSet/istio-ingressgateway-565d647f8c Containers: istio-proxy: Image: docker.io/istio/proxyv2:1.10.3 Ports: 15021/TCP, 8080/TCP, 8443/TCP, 15090/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP Args: proxy router --domain $(POD_NAMESPACE).svc.cluster.local --proxyLogLevel=warning --proxyComponentLogLevel=misc:error --log_output_level=default:info --serviceCluster istio-ingressgateway Limits: cpu: 2 memory: 1Gi Requests: cpu: 100m memory: 128Mi Readiness: http-get http://:15021/healthz/ready delay=1s timeout=1s period=2s #success=1 #failure=30 Environment: JWT_POLICY: third-party-jwt PILOT_CERT_PROVIDER: istiod CA_ADDR: istiod.istio-system.svc:15012 NODE_NAME: (v1:spec.nodeName) POD_NAME: istio-ingressgateway-565d647f8c-cxwfx (v1:metadata.name) POD_NAMESPACE: istio-system (v1:metadata.namespace) INSTANCE_IP: (v1:status.podIP) HOST_IP: (v1:status.hostIP) SERVICE_ACCOUNT: (v1:spec.serviceAccountName) CANONICAL_SERVICE: (v1:metadata.labels[' service.istio.io/canonical-name']) CANONICAL_REVISION: (v1:metadata.labels[' service.istio.io/canonical-revision']) ISTIO_META_WORKLOAD_NAME: istio-ingressgateway ISTIO_META_OWNER: kubernetes://apis/apps/v1/namespaces/istio-system/deployments/istio-ingressgateway ISTIO_META_MESH_ID: cluster.local TRUST_DOMAIN: cluster.local ISTIO_META_UNPRIVILEGED_POD: true ISTIO_META_ROUTER_MODE: standard ISTIO_META_CLUSTER_ID: Kubernetes Mounts: /etc/istio/config from config-volume (rw) /etc/istio/ingressgateway-ca-certs from ingressgateway-ca-certs (ro) /etc/istio/ingressgateway-certs from ingressgateway-certs (ro) /etc/istio/pod from podinfo (rw) /etc/istio/proxy from istio-envoy (rw) /var/lib/istio/data from istio-data (rw) /var/run/secrets/istio from istiod-ca-cert (rw) /var/run/secrets/kubernetes.io/serviceaccount from istio-ingressgateway-service-account-token-wxgjk (ro) /var/run/secrets/tokens from istio-token (ro) Conditions: Type Status PodScheduled False Volumes: istiod-ca-cert: Type: ConfigMap (a volume populated by a ConfigMap) Name: istio-ca-root-cert Optional: false podinfo: Type: DownwardAPI (a volume populated by information about the pod) Items: metadata.labels -> labels metadata.annotations -> annotations limits.cpu -> cpu-limit requests.cpu -> cpu-request istio-envoy: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: istio-data: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: istio-token: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 43200 config-volume: Type: ConfigMap (a volume populated by a ConfigMap) Name: istio Optional: true ingressgateway-certs: Type: Secret (a volume populated by a Secret) SecretName: istio-ingressgateway-certs Optional: true ingressgateway-ca-certs: Type: Secret (a volume populated by a Secret) SecretName: istio-ingressgateway-ca-certs Optional: true istio-ingressgateway-service-account-token-wxgjk: Type: Secret (a volume populated by a Secret) SecretName: istio-ingressgateway-service-account-token-wxgjk Optional: false QoS Class: Burstable Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message

Warning FailedScheduling 19s (x22 over 13m) default-scheduler 0/3 nodes are available: 1 Too many pods, 1 node(s) had taint {sonic-node-group: api}, that the pod didn't tolerate, 1 node(s) had taint {sonic-node-group: nff}, that the pod didn't tolerate.

On Friday, October 1, 2021, 12:56:50 PM PDT, John Howard @.***> wrote:

I don't think email attachments work on github.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/istio/istio/issues/35431#issuecomment-932543841, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXNJBZZGI5IHAGEHWQ3UEYNX3ANCNFSM5FDEZOKA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

istio-policy-bot commented 2 years ago

🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2021-10-01. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions.

Created by the issue and PR lifecycle manager.