search

cloudflare / cloudflare-ingress-controller

A Kubernetes ingress controller for Cloudflare's Argo Tunnels
Apache License 2.0
364 stars 55 forks source link

Unable to configure argo on minikube #146

Open sashok2k opened 5 years ago

sashok2k commented 5 years ago

Hi

I'm sure i'm missing something during configuration of Argo Ingress for my minikube installation, but i'm getting following errors after third reinstall of whole configuration:

time="2019-01-12T16:15:09Z" level=error msg="link exited with error (*net.DNSError) 'lookup _warp._tcp.cloudflarewarp.com on 10.96.0.10:53: no such host', repairing ..." hostname=mk-alex.actonica.ru origin="echo.default:80"

time="2019-01-12T16:15:09Z" level=info msg="link repair starts in 23.014208ms" hostname=mk-alex.actonica.ru origin="echo.default:80"

time="2019-01-12T16:15:09Z" level=info msg="ResolveEdgeIPs err"

time="2019-01-12T16:15:09Z" level=error msg="link exited with error (*net.DNSError) 'lookup _warp._tcp.cloudflarewarp.com on 10.96.0.10:53: no such host', repairing ..." hostname=mk-alex.actonica.ru origin="echo.default:80"

time="2019-01-12T16:15:09Z" level=info msg="link repair starts in 25.343819ms" hostname=mk-alex.actonica.ru origin="echo.default:80"

time="2019-01-12T16:15:09Z" level=info msg="ResolveEdgeIPs err"

In my case i'm configuring subdomain, so i follow Argo Tunnels for Subdomains article.

rm -rf ~/.minikube

minikube start
helm init
helm repo update   

helm install --name anydomain --namespace default \
    --set rbac.create=true \
    --set controller.ingressClass=argo-tunnel \
    --set controller.logLevel=6 \
    cloudflare/argo-tunnel

kubectl create secret generic actonica.ru --from-file="$HOME/.cloudflared/cert.pem"

awk '/BEGIN.*TUNNEL/{mark=1}/END.*TUNNEL/{print;mark=0}mark' ~/.cloudflared/cert.pem >> mk-alex.pem
kubectl create secret generic mk-alex.actonica.ru --from-file="mk-alex.pem"

kubectl apply -f argo-sample.yaml 
kubectl apply -f argo-tunnel-sample.yaml

argo-sample.yaml.txt argo-tunnel-sample.yaml.txt

I could miss some steps that I did...

What i'm doing wrong?

ntfrnzn commented 5 years ago

@sashok2k by chance I ran into this exact problem a couple days ago and it took a while to make progress figuring it out. It's an issue with minikube configuration.

The cloudflared process needs to do an src lookup, the equivalent of dig SRV _warp._tcp.cloudflarewarp.com

If you try this on a container running on your minikube docker daemon, 

eval $(minikube docker-env)
docker run --rm -it sequenceiq/alpine-dig dig SRV _warp._tcp.cloudflarewarp.com

you'll see it fails.

I think that the latest minikube release changed the dns configuration ... becuase of the move to coredns? Minikube misconfiguration leaves /etc/resolve.conf pointing at 10.0.2.3.

I'm not sure of the "correct" solution, or whether to file an issue against minikube, but to resolve your problem in the short term: minikube ssh, edit /etc/systemd/resolved.conf to add DNS=8.8.8.8 and systemctl restart systemd-resolved

If this is an issue for cloudflare-ingress-controller, it's in the category of "improve error messages"

ntfrnzn commented 5 years ago

After thinking about it a little, I think it could be an issue against https://github.com/cloudflare/cloudflared, i.e. "LookupSrv should use 1.1.1.1 instead of DefaultResolver" and avoid bad local dns misconfiguration. But the cloudflare people can decide themselves whether that's what they want.

sashok2k commented 5 years ago

@ntfrnzn I did as you suggested, but that didn't helped... dig still don't return correct response. I'm thinking to change coredns configuration and have required records inside cluster. Will do that when I have time. Switched to nginx for now...