compumike / hairpin-proxy

PROXY protocol support for internal-to-LoadBalancer traffic for Kubernetes Ingress users. If you've had problems with ingress-nginx, cert-manager, LetsEncrypt ACME HTTP01 self-check failures, and the PROXY protocol, read on.
MIT License
561 stars 61 forks source link

Not able to make it work with microk8s ingress(nginx) #8

Open jeremybusk opened 3 years ago

jeremybusk commented 3 years ago

First of all, this is awesome project! Thank you for adding this. I have an issue though because I want to use microk8s.

ingress-nginx-controller.ingress-nginx.svc.cluster.local resolution is not working in on the microk8s installation of kubernetes. Any idea of what this should be or how to get it to work? Not sure what I should change it to on microk8s. I'm a little new to kubernetes. Thanks for they help!

Here is more info for debug.

microk8s enable ingress

microk8s kubectl get pods -n hairpin-proxy NAME READY STATUS RESTARTS AGE hairpin-proxy-controller-7b48d47458-jnspd 1/1 Running 0 16h hairpin-proxy-haproxy-5957c6fdc-k8rnv 0/1 CrashLoopBackOff 198 16h

microk8s kubectl logs hairpin-proxy-haproxy-5957c6fdc-k8rnv -n hairpin-proxy [NOTICE] 044/183415 (1) : haproxy version is 2.2.4-de45672 [ALERT] 044/183415 (1) : parsing [/usr/local/etc/haproxy/haproxy.cfg:7] : 'server my_server' : could not resolve address 'ingress-nginx-controller.ingress-nginx.svc.cluster.local'. [ALERT] 044/183415 (1) : parsing [/usr/local/etc/haproxy/haproxy.cfg:16] : 'server my_server' : could not resolve address 'ingress-nginx-controller.ingress-nginx.svc.cluster.local'. [ALERT] 044/183415 (1) : Failed to initialize server(s) addr.

and even more from services root@ku1:~/cert# k get svc --all-namespaces NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default kubernetes ClusterIP 10.152.183.1 443/TCP 20h kube-system kube-dns ClusterIP 10.152.183.10 53/UDP,53/TCP,9153/TCP 20h default httpbin ClusterIP 10.152.183.148 8080/TCP 17h hairpin-proxy hairpin-proxy ClusterIP 10.152.183.220 80/TCP,443/TCP 16h default cm-acme-http-solver-qtrpl NodePort 10.152.183.161 8089:31375/TCP 16h cert-manager cert-manager ClusterIP 10.152.183.21 9402/TCP 15h cert-manager cert-manager-webhook ClusterIP 10.152.183.200 443/TCP 15h default web NodePort 10.152.183.221 8080:31771/TCP 3h14m default test NodePort 10.152.183.81 80:30208/TCP 162m default test2 NodePort 10.152.183.128 8080:32560/TCP 153m

jeremybusk commented 3 years ago

Got pods up but probably not in the right way. Still having issues curling my host through hairpin-proxy

kubectl get svc -n ingress | grep ingress output: nginx-ingress-microk8s-controller-86rjl.ingress.svc.cluster.local

kubectl edit deploy hairpin-proxy-haproxy -n hairpin-proxy
# Modify with below
   spec:
      containers:
      - env:
        - name: TARGET_SERVER
          value: nginx-ingress-microk8s-controller-86rjl.ingress.svc.cluster.local
        image: compumike/hairpin-proxy-haproxy:0.1.2

Expose controller

kubectl expose pod nginx-ingress-microk8s-controller-86rjl --type=NodePort --port=8080 -n ingress

Pods are up now

root@km1:~/src/hairpin-proxy# k get pods -n hairpin-proxy -o wide
NAME                                        READY   STATUS    RESTARTS   AGE    IP             NODE   NOMINATED NODE   READINESS GATES
hairpin-proxy-controller-7b48d47458-xlzrt   1/1     Running   5          174m   10.1.132.196   km1    <none>           <none>
hairpin-proxy-haproxy-5c8b4c9cf-84n6j       1/1     Running   0          19m    10.1.132.205   km1    <none>           <none>
$ kubectl get configmap -n kube-system coredns -o=jsonpath='{.data.Corefile}'
.:53 {
    rewrite name minimal.uvoo.io hairpin-proxy.hairpin-proxy.svc.cluster.local # Added by hairpin-proxy
    errors
    health {
      lameduck 5s

Let's try and see if it works

root@km1:~/src/hairpin-proxy# k get svc --all-namespaces | grep hair
hairpin-proxy   hairpin-proxy                             ClusterIP   10.152.183.106   <none>        80/TCP,443/TCP           165mm

/ # host minimal.uvoo.io
minimal.uvoo.io has address 10.152.183.106

Works from outside host
~$ curl -L -k -H "Host: minimal.uvoo.io" https://km1
Hello, world!
Version: 2.0.0
Hostname: exampleservice-7b8db6b8b-qs6mz

but curl hangs in test container
curl -k http://minimal.uvoo.io

I'm still missing something though I admit that I don't know exactly what I'm doing. Hoping someone else can point me in the right direction.

ozby commented 2 years ago

change value: nginx-ingress-microk8s-controller-86rjl.ingress.svc.cluster.local to value: ingress.ingress.svc.cluster.local