search

kubernetes-sigs / kind

Kubernetes IN Docker - local clusters for testing Kubernetes
https://kind.sigs.k8s.io/
Apache License 2.0
13.47k stars 1.56k forks source link

Load balancer external-ip pending #411

Closed jteng closed 5 years ago

jteng commented 5 years ago

I deploy the same deployment and expose it via the same kubectl expose deployment supervisor-deployment --type=LoadBalancer --name=supervisor-service command, in docker-desktop, a service is created successfully with external-ip assigned, but in kubernetes-admin@kind cluster, the service is created with external-ip pending forever.

here is the output from kubectl get svc on docker-desktop: supervisor-service LoadBalancer 10.100.184.32 localhost 9090:31778/TCP 13s while the kind cluster: supervisor-service LoadBalancer 10.111.158.50 <pending> 9090:31054/TCP 16m

BenTheElder commented 5 years ago

kind does not support type=LoadBalancer currently, load balancers are cloud provider specific. see #99 for some discussion around this.

jteng commented 5 years ago

thanks @BenTheElder

aojea commented 5 years ago

kind does not support type=LoadBalancer currently, load balancers are cloud provider specific. see #99 for some discussion around this.

@BenTheElder should we create a kind load balancer? seems a nice and funny project

PercyLau commented 5 years ago

@aojea I agree. It would be pretty interesting if we could test novel load balancers in our own desktop. Besides, some distributed machine learning infrastructure, e.g., kubeflow, require an external load balancer to work with.

BenTheElder commented 5 years ago

You can deploy a loadbalancer with kind of course, but the integrated Kubernetes object is typically only supported "in the cloud" and Kubernetes does not ship one.

On docker for Mac in particular this is not quite so trivial.

Kubeflow should be able to work fine with something like a nodePort or an ingress, I will speak to them.

BenTheElder commented 5 years ago

Also: https://github.com/kubeflow/website/pull/861#issuecomment-510555085

RothAndrew commented 4 years ago

For others who stumble upon this (as I did), I've had nice success using Inlets to get a public IP for LoadBalancer services.

k-nuhdim commented 4 years ago

f you are not using GCE or EKS (you used kubeadm) you can add an externalIPs spec to your service YAML. You can use the IP associated with your node's primary interface such as eth0. You can then access the service externally, using the external IP of the node.

... spec: type: LoadBalancer externalIPs:

BenTheElder commented 4 years ago

On linux you can deploy metallb, but on mac / windows docker containers are not reachable from the host (only via a special port forwarding mechanism). See also: https://kind.sigs.k8s.io/docs/user/ingress/ https://kind.sigs.k8s.io/docs/user/resources/#how-to-use-kind-with-metalllb https://docs.docker.com/docker-for-mac/networking/#i-cannot-ping-my-containers https://github.com/docker/for-mac/issues/2670

ghost commented 3 years ago

Docker Desktop on macOS users can use TunTap with MetalLB. Here's an example that feedback shows still works on macOS as of at least Catalina: https://github.com/AlmirKadric-Published/docker-tuntap-osx

I've also seen some interesting solutions using dnsmasq. K3s with k3d also ships with its own load balancer which can be adjusted at cluster creation time and multiple clusters are supported.

BenTheElder commented 3 years ago

see #1961, which also brings up tuntap. we're going to start by writing down the existing options.

BenTheElder commented 3 years ago

The k3d "loadbalancer" implementation (1) is more or less the same thing KIND is doing is doing in HA mode. You still need to manually specify port mapping at cluster creation time, which is more or less analagous to https://kind.sigs.k8s.io/docs/user/configuration/#extra-port-mappings

Without something like tuntap there isn't a good way to have arbitrary mappings at runtime on mac / windows, since the containers are just not routable (only the port forwarding, which is some vpnkit magic specific to proxying ports).

On linux you can just deploy metallb and move on. Metallb CI uses kind. But then any developers on mac / windows are pretty SOL.

ghost commented 3 years ago

I found a good tutorial for getting EXTERNAL-IP to assign dynamically from a.pool using MetalLB on Mac. It's using TunTap and works on macOS 11. Because Linux is (and should be) the primary development platform, this gives as least some reprieve to those feeling the pain on macOS: https://www.thehumblelab.com/kind-and-metallb-on-mac/

BenTheElder commented 3 years ago

I haven't had a chance to try this yet myself but it looks good! Previously https://github.com/kubernetes-sigs/kind/issues/1961#issuecomment-738977020, also reached out to OP on twitter 😅

https://twitter.com/Codydearkland/status/1334946807202238464?s=19 🧵

I think at the very least this option needs to be referenced in our docs #1961