search

istio / istio.io

Source for the istio.io site
https://istio.io/
Apache License 2.0
762 stars 1.53k forks source link

[Testing] Istio-ingressgateway unable to obtain external IP address #8353

Open nmittler opened 3 years ago

nmittler commented 3 years ago

This was discovered while working on https://github.com/istio/istio.io/pull/8346.

As part of this work, I tried modifying the custom logic for IP address fields in __cmp_like to not allow matching against (pending) or (none). Apparently this was added to support node port gateways, which should no longer be needed now that the test framework uses MetalLB with kind.

All tests pass except for one that uses the default profile:

The error:

VERIFY FAILED snip_determining_the_ingress_ip_and_ports_1 (timeout after 120s): received: "NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                                      AGE
istio-ingressgateway   LoadBalancer   10.96.160.174   <pending>     15021:30127/TCP,80:31667/TCP,443:30820/TCP,15443:30159/TCP,15012:32392/TCP,15017:31260/TCP,31400:32655/TCP   15m", expected: "NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)   AGE
istio-ingressgateway   LoadBalancer   172.21.109.129   130.211.10.121   ...       17h"

So after 15 minutes of being up, istio-ingressgateway was never assigned an external IP address. It's not immediately clear why this would be.

The multicluster tests (which script the istio installation, rather than using the Istio test framework to install) perform a similar verification of he istio-eastwestgateway and it works fine.

nmittler commented 3 years ago

FYI @frankbu @ericvn @stevenctl

frankbu commented 3 years ago

All of the other single-cluster tests that use the ingress gateway use similar code to determine if it has an ext LB, so I wonder if this implies that all the single cluster tests are using nodeport to access the gateway?

What's the difference between the single and multicluster test clusters?

ericvn commented 3 years ago

Need to investigate more, but I believe kind by itself would use nodeport. Adding MetalLB would be a way to use the external load balancer. Will this cause any issues trying to run the tests locally on a Mac for example?

nmittler commented 3 years ago

@ericvn

Will this cause any issues trying to run the tests locally on a Mac for example?

If you're running with the same kind environment locally, I would expect the same behavior

nmittler commented 3 years ago

@frankbu

What's the difference between the single and multicluster test clusters?

I haven't looked closely at the multicluster setup for kind, and I suspect the investigation should probably start there. As far as installing the control plane, the tests that use the default and demo profiles are getting istio installed by the istio test framework, which may be slightly different than what happens when just using a standard istioctl install.

frankbu commented 3 years ago

One thing that this highlights is that even though the doc in question has instructions for both ext LB and nodeport cases, we really are only testing one or the other. I had assumed we were testing with the LB env, but for some reason we are using nodeport. Once we figure out why, we will still have the problem that we don't test both paths. Maybe we really need setup config choices for both with and without LB, so we can run the test both ways.

That said, it's probably not really worth the extra effort. Ideally we test with LB IP only, the nodeport is just for simple getting started demos on clusters like minikube, so I'd think we will find out soon enough if it's not working, even if we don't actually check that case in the test.

frankbu commented 3 years ago

@ericvn, I think you may be right. Basic kind cluster has no LB, and that must we what we use in the single cluster setups. Multicluser adds the LB using MetalLB.

shahabedinh commented 1 year ago

Have the same issue. I deployed k3s on ec2, removed Traefik, and installed istio . From the two ingress, one ( istio-ingressgatewa) got external IP, but the second one (stio-eastwestgateway) has pending for external IP

service/istio-ingressgateway    LoadBalancer   10.43.153.214   10.0.12.55,10.0.3.72,10.0.5.69   15021:32733/TCP,80:31305/TCP,443:30329/TCP                        66m

service/istio-eastwestgateway   LoadBalancer   10.43.1.82      <pending>                        15021:30951/TCP,15443:30170/TCP,15012:30732/TCP,15017:31185/TCP   64m