search

kubernetes / examples

Kubernetes application example tutorials
Apache License 2.0
6.16k stars 4.52k forks source link

Getting DNS errors in example guestbook-go, involving failure to resolve "redis_slave" #511

Closed dkwgit closed 11 months ago

dkwgit commented 11 months ago

UPDATE. I completely missed Antoine Toussaint's prior issue 506, AntoineToussaint. He diagnosed the same things. His proposed fix is the better one, however, my one line fix to the DNS issue, see #5 below, gets things going quickly. Like he found, I also did also need to make a slight adjustment to the go import to correctly import simpleredis. It needs to be imported as: github.com/xyproto/simpleredis/v2

Original Issue: It's possible this issue is a bug in the guestbook-go example. Or, maybe it is just a problem with Docker Desktop K8s, which is what i was using. I don't know. I'm creating the issue to possibly help others who run into the same problem.

  1. The go program in guestbook-go/main.go shows this panic after adding list items in the guestbook (and the items never show in the web app):

    PANIC: dial tcp: lookup redis-slave on 10.96.0.10:53: server misbehaving
goroutine 63 [running]:
github.com/codegangsta/negroni.(*Recovery).ServeHTTP.func1(0x7f954fac7b58, 0xc82007f100, 0xc8200ecd20)
    /go/src/github.com/codegangsta/negroni/recovery.go:34 +0xe9
panic(0x7a8c60, 0xc820122f00)
    /usr/local/go/src/runtime/panic.go:426 +0x4e9
main.HandleError(0x6e2780, 0xc8201a5980, 0x7f954fa7e0a8, 0xc820122f00, 0x0, 0x0)
    /go/src/github.com/GoogleCloudPlatform/kubernetes/examples/guestbook-go/_src/main.go:71 +0x59
main.ListRangeHandler(0x7f954fac7b58, 0xc82007f100, 0xc8200f87e0)

  2. Using dnsutils via kubectl exec -i -t dnsutils -- nslookup redis-slave shows:

    Server:         10.96.0.10
Address:        10.96.0.10#53

** server can't find redis-slave: SERVFAIL

command terminated with exit code 1

  3. Logs from kube-dns, via kubectl logs --namespace=kube-system -l k8s-app=kube-dns show:

    [INFO] 10.1.0.67:35518 - 32059 "A IN redis-slave.svc.cluster.local. udp 47 false 512" NXDOMAIN qr,aa,rd 140 0.000136801s
[INFO] 10.1.0.67:48817 - 11700 "AAAA IN redis-slave.cluster.local. udp 43 false 512" NXDOMAIN qr,aa,rd 136 0.0000953s
[INFO] 10.1.0.67:34640 - 708 "A IN redis-slave.cluster.local. udp 43 false 512" NXDOMAIN qr,aa,rd 136 0.0000894s
[INFO] 10.1.0.67:33927 - 62058 "A IN redis-slave. udp 29 false 512" - - 0 2.001422668s
[ERROR] plugin/errors: 2 redis-slave. A: read udp 10.1.0.50:44112->192.168.65.7:53: i/o timeout

  4. However, redis-master (from redis-master-service.yaml), does resolve (i.e. DNS is working, in some fashion), per kubectl exec -i -t dnsutils -- nslookup redis-master

    Server:         10.96.0.10
Address:        10.96.0.10#53

Name:   redis-master.default.svc.cluster.local
Address: 10.109.231.172

  5. A one line change fixes this for me. (Not saying this is the right thing to do for the whole example for everyone--I'd have to research more--but it might help other Docker Desktop K8S users to get the example working, if they are experiencing the same problem).

    If I change the service name in redis-replica-service.yaml on line 4 to name: redis-slave

    the web app starts working (displaying added items) and the DNS errors stop.

  6. Additional Info on versions/systems I used and changes I made to the example. (I have not tried this with a K8S other than Docker Desktop's built-in).