added simple readyz check

[networkop]

fixes #117

[networkop]

@onedr0p can you check if this PR works?

[onedr0p]

I just wanted to let you know I see this but haven't had time to test.

[onedr0p]

@networkop I still haven't had time to test this but have you noticed with your changes that...

a) k8s_gateway starts and serves non-cluster ingress DNS without the cluster being online and; b) without restarting k8s_gateway and your cluster becomes available that k8s_gateway is able to serve cluster ingress DNS?

[onedr0p]

I have found a different way to use k8s_gateway that's much better and doesn't require me to deploy k8s_gateway outside my cluster anymore. Thanks for looking into this though, maybe it still provides value?

[networkop]

cool. I'll leave it open for now

[solidDoWant]

I'm currently doing what onedr0p did (in fact my DNS setup is based of the way his used to be). Is there a build for this that I can test somewhere? I can build it myself, but figured if I'm going to test this then you might prefer that I an "official" build in case any build-specific issues cause a false negative or positive while testing.

[networkop]

@solidDoWant I don't think there's a build, I need to enable that for PRs. You can have a look at the CI files to see how the build is produced

[solidDoWant]

Got it built via CGO_ENABLED=0 GOOS=freebsd GOARCH=amd64 go build cmd/coredns.go and running. I'm testing by setting the cluster server to a valid IP on my network, but not one that is running Kubernetes.

Here's a log, with my normal DNS lookups ommitted:

[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[INFO] plugin/auto: Inserting zone `10.in-addr.arpa.' from: /usr/local/etc/coredns/zones/10.in-addr.arpa
[INFO] plugin/auto: Inserting zone `echozulu.local.' from: /usr/local/etc/coredns/zones/echozulu.local
.:53 on 10.1.0.1
.:53 on 10.0.0.1
.:53 on 10.2.0.1
.:53 on 10.3.0.1
.:53 on 10.4.0.1
.:53 on 10.5.0.1
.:53 on ::1
.:53 on 127.0.0.1
CoreDNS-1.9.3+k8s_gateway-0.3.0
freebsd/amd64, go1.21.0,
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[ERROR] Recovered from panic in server: "dns://10.2.0.1:53" runtime error: invalid memory address or nil pointer dereference
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[ERROR] Recovered from panic in server: "dns://10.3.0.1:53" runtime error: invalid memory address or nil pointer dereference
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[INFO] plugin/k8s_gateway: api-server not ready: "Get \"https://10.143.0.1:443/readyz\": dial tcp 10.143.0.1:443: i/o timeout", retrying
[INFO] plugin/k8s_gateway: Waiting for api-server to become ready

10.3.0.1 is one of the end user devices on my network (the desktop that I'm typing this on), and 10.143.0.1 is the IP address of a non-existent Kubernetes API server. As you can see from the logs, there are a lot of Recovered from panic in server whenever a DNS request is sent by my computer. When the Kubernetes API service is reachable, everything works properly:

[INFO] plugin/k8s_gateway: Waiting for api-server to become ready
[INFO] plugin/auto: Inserting zone `10.in-addr.arpa.' from: /usr/local/etc/coredns/zones/10.in-addr.arpa
[INFO] plugin/auto: Inserting zone `echozulu.local.' from: /usr/local/etc/coredns/zones/echozulu.local
.:53 on 127.0.0.1
.:53 on 10.1.0.1
.:53 on 10.0.0.1
.:53 on 10.2.0.1
.:53 on 10.3.0.1
.:53 on 10.4.0.1
.:53 on 10.5.0.1
.:53 on ::1
CoreDNS-1.9.3+k8s_gateway-0.3.0
freebsd/amd64, go1.21.0,
[INFO] plugin/k8s_gateway: api-server ready, proceeding
[INFO] plugin/k8s_gateway: Building k8s_gateway controller
W1028 11:20:35.630211   92746 warnings.go:70] Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
[INFO] plugin/k8s_gateway: access to `gateway.networking.k8s.io` is forbidden, please check RBAC. Not syncing GatewayAPI resources.
W1028 11:20:35.633950   92746 warnings.go:70] Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
[INFO] plugin/k8s_gateway: access to `k8s.nginx.org/v1` is forbidden, please check RBAC. Not syncing VirtualServer resources.
[INFO] plugin/k8s_gateway: Starting k8s_gateway controller
[INFO] plugin/k8s_gateway: Waiting for controllers to sync
W1028 11:20:35.643402   92746 warnings.go:70] Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
W1028 11:20:35.656271   92746 warnings.go:70] Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
W1028 11:20:35.664753   92746 warnings.go:70] Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
W1028 11:20:35.691102   92746 warnings.go:70] Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
[INFO] plugin/k8s_gateway: Synced all required resources

This PR is good enough for me as is, but it would be nice if the nil pointer dereference could be fixed.

Let me know if you need more info on my setup, and thanks for your work on this!

[solidDoWant]

I just merged master into this branch and rebuilt. Same issue on both this branch (as is), and this branch on top of d72591c331765703951533b21b49e98a8152d5fd.

[networkop]

@solidDoWant can you enable debug to see where exactly that nil pointer error comes from?