xetys / hetzner-kube

A CLI tool for provisioning kubernetes clusters on Hetzner Cloud
Apache License 2.0
745 stars 116 forks source link

How do I get an external address attached to the ingress controller #103

Closed derryx closed 6 years ago

derryx commented 6 years ago

Hi!

I am quite new to Kubernetes and I might ask a stupid question. So please forgive me.

I am able to create a cluster and deploying helm charts in that cluster works fine. But when I install an ingress controller (I want to install Jenkins-X) I always end up with the external address in state "pending". What do I need to do to get an external address attached? Jenkins-X does not proceed with its installation as long there is no external address.

Thanks Thomas

xetys commented 6 years ago

There are two ways of exposing services, LoadBalancer service type, and ingress. You are asking for LoadBalancer services. This means, a cloud provider takes a floating IP from the cloud infrastructures and connects to your cluster. This is a feature, which most commonly exists only in cloud kubernetes clusters, like on GCP, AWS or Azure. There also a custom one inside rancher. \

For the rest you use ingress. The ingress means, you declare an additional resource called Ingress which takes a service and a host name and makes the binding. You need a ingress controller to do this in real. For this purpose, you might use nginx ingress controller. You have one in the hetzner-kube admin section.

A quick google lead me to an obvious issue you ran into: as of http://jenkins-x.io/commands/jx_install/ jenkins-x automatically installs an nginx ingress controller for you. But the default ingress controller tries to get its IP over an external IP from a load balancer. That is not the way we do this in custom clusters like hetzner-kube. Here the ingress controller must be deployed as DaemonSets, running on every (specified) node with hostNetwork on. In order to use jx with a hetzner-kube cluster, you must use --skip-ingress and install hetzner-kube cluster addon install ingress -n your-cluster in advance.

I didn't test this, but this should work. Please close this issue if your issue is resolved.

derryx commented 6 years ago

Unfortunately I am stuck here now:

thomas@taurus:~$ kubectl get pods -w --all-namespaces
NAMESPACE     NAME                                                     READY     STATUS             RESTARTS   AGE
default       ingress-nginx-ingress-controller-7fv2k                   1/1       Running            0          6m
default       ingress-nginx-ingress-controller-cqzwq                   1/1       Running            0          6m
default       ingress-nginx-ingress-default-backend-559c4689b4-jrr5g   1/1       Running            0          6m
jx            jenkins-856f48dbd4-xwzrb                                 0/1       Pending            0          4m
jx            jenkins-x-chartmuseum-5bfc74798d-8p8jx                   0/1       Pending            0          4m
jx            jenkins-x-docker-registry-f5f9b7bc9-48txv                0/1       Pending            0          4m
jx            jenkins-x-heapster-c6d44bc95-qsxps                       2/2       Running            0          4m
jx            jenkins-x-mongodb-77c6dc684c-k9bmf                       0/1       Pending            0          4m
jx            jenkins-x-monocular-api-57554664c7-sqs4j                 0/1       CrashLoopBackOff   4          4m
jx            jenkins-x-monocular-prerender-64f989797b-hgmch           1/1       Running            0          4m
jx            jenkins-x-monocular-ui-77994894d4-hjghr                  1/1       Running            0          4m
jx            jenkins-x-nexus-7995877df5-m6fgx                         0/1       Pending            0          4m
jx            pipelinecontroller-55d65ccc6-6646z                       1/1       Running            0          4m
kube-system   etcd-jenkinsx-cluster-master-01                          1/1       Running            0          8m
kube-system   hcloud-cloud-controller-manager-7979447495-jhffh         1/1       Running            0          8m
kube-system   kube-apiserver-jenkinsx-cluster-master-01                1/1       Running            0          7m
kube-system   kube-controller-manager-jenkinsx-cluster-master-01       1/1       Running            0          7m
kube-system   kube-dns-86f4d74b45-bjc7n                                3/3       Running            0          8m
kube-system   kube-flannel-ds-62xvg                                    1/1       Running            0          8m
kube-system   kube-flannel-ds-8gnzg                                    1/1       Running            0          8m
kube-system   kube-flannel-ds-m2vmj                                    1/1       Running            0          8m
kube-system   kube-proxy-g5rbr                                         1/1       Running            0          8m
kube-system   kube-proxy-kjtjm                                         1/1       Running            0          8m
kube-system   kube-proxy-vqmlt                                         1/1       Running            0          8m
kube-system   kube-scheduler-jenkinsx-cluster-master-01                1/1       Running            0          7m
kube-system   tiller-deploy-84f4c8bb78-txgrt                           1/1       Running            0          6m
rook-system   rook-agent-2n746                                         1/1       Running            0          7m
rook-system   rook-agent-mctz5                                         1/1       Running            0          7m
rook-system   rook-operator-5c89ff9496-vw25m                           1/1       Running            0          7m
rook          rook-ceph-mgr0-578fdfd75b-rphqt                          1/1       Running            0          6m
rook          rook-ceph-mon0-fq7pt                                     1/1       Running            0          7m
rook          rook-ceph-mon1-x7njt                                     1/1       Running            0          6m
rook          rook-ceph-mon2-htcpb                                     1/1       Running            0          6m
rook          rook-ceph-osd-6nhx9                                      1/1       Running            0          6m
rook          rook-ceph-osd-7jzfb                                      1/1       Running            0          6m

I can access the ingress controller on the worker nodes. So this is working fine. Maybe this is more an issue for the Jenkins X folks now.

xetys commented 6 years ago

yeah, really looks like a JX issue here.

derryx commented 6 years ago

This seems to be an Jenkins X problem. Closing issue.