search

postfinance / kubenurse

Kubernetes network monitoring
MIT License
407 stars 39 forks source link

Non-standard cluster domain #158

Open jone opened 2 weeks ago

jone commented 2 weeks ago

Hi

I've tried to install kubenurse on exoscale SKS. Exoscale SKS modifies the standard kubernetes cluster domain to be [uuid].cluster.local.

This causes this error:

2024/08/31 10:47:22 ERROR request failure in httptrace event_type=round_trip_error request_type=api_server_dns err="dial tcp: lookup kubernetes.default.svc.cluster.local on 10.96.0.10:53: no such host"
2024/08/31 10:47:27 ERROR request failure in httptrace event_type=dns_done request_type=api_server_dns err="lookup kubernetes.default.svc.cluster.local on 10.96.0.10:53: no such host"

The DNS entry kubernetes.default.svc.cluster.local does not exist; it should be something like kubernetes.default.svc.4ea3d352-64ed-41f7-ba54-3d181c211d19.cluster.local.

This value seems to be hard coded here: https://github.com/postfinance/kubenurse/blob/754b407ce768dc220984b8cb8afcba2a4f1d5c11/internal/servicecheck/servicecheck.go#L155

Is it possible to make that configurable or use another strategy to find the correct domain? I'm not very familiar with the k8s internas, but I would assume that the API domain can be looked up somewhere.

The kubenurse service domain is already configurable and when I understand it correctly, I can configure it in helm with the service_url value.

Exoscale has plans to switch to the standard domain, but this is only for newly created clusters. I could also circumvent the problem by adding coredns alias entries, but having it configurable would be more convenient and less intrusive, I think 😉

clementnuss commented 3 days ago

hi @jone

thanks for raising this issue. I checked a little, I think it would be possible to derive the cluster domain from the search domains in /etc/resolv.conf, but I think it could be too easy to break 😅

I will implement an env variable that permits setting it!

clementnuss commented 3 days ago

can you try with this container ? https://github.com/users/clementnuss/packages/container/kubenurse/274375776?tag=issue-158-fix

you can set the DNS name of the APIServer with the KUBERNETES_SERVICE_DNS env var