search

postfinance / kubenurse

Kubernetes network monitoring
MIT License
416 stars 39 forks source link

404 Not Found service endpoints #33

Closed eli-halych closed 11 months ago

eli-halych commented 2 years ago 
"user_agent": "curl/7.76.1",
 "request_uri": "/alive",
 "remote_addr": "127.0.0.1:38864",
 "api_server_direct": "ok",
 "api_server_dns": "ok",
 "me_ingress": "404 Not Found",
 "me_service": "Get \"http://kubenurse.kube-system.svc.cluster.local:8080/alwayshappy\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)",
 "neighbourhood_state": "ok",

Any idea why /alive and other endpoints return 404 when trying to send requests to the service?

djboris9 commented 2 years ago

Do you have any NetworkPolicies or similar in place, which are causing the timeout? Are other services reachable as usual (=kube-proxy works)?

P.S. When the me_service works, you're a step further and propably the me_ingress will work then too

eli-halych commented 2 years ago

34 fixed my problem with me_service - the endpoint became available and it is in the OK state now.

Regarding me_ingress, I am still working on it, because I am using traefik instead of nginx.

djboris9 commented 2 years ago

at least the ingress.className should be set to a matching one for traefik

eli-halych commented 2 years ago

Is there any documentation or an example how to enable traefik ingress routing for kubenurse? The idea is to use mydummyurl.com/kubenurse path within the Kubernetes cluster.

eli-halych commented 2 years ago

Alright, I noticed it is actually working when I check the /metrics endpoint. And all the pods are "turning on/off" meaning readiness probes are succeeding, then failing and then it's on repeat. No restarts - just readiness probes failing chaotically. But the /metrics endpoint is stable overall.

djboris9 commented 2 years ago

Could you please share your Helm values you are using to deploy the kubenurse? So I can deploy it with the same configuration and have a better understanding what is failing.

eli-halych commented 2 years ago

I solved it with this MR #38 - a new chart designed for Traefik version 2.x that allows access to Kubenurse via the path example.com/kubenurse/* .

This was my case. And since I used Traefik 2.x, I switched Ingress resource and backend/frontend logic to the one suitable for version 2.x - Middlewares and IngressRoutes.

eli-halych commented 2 years ago

@djboris9, what do you think about this MR #38?

clementnuss commented 11 months ago

@eli-halych cf my comment in #38.

closing this issue, as it has more to do with Traefik Ingress Controller peculiarities than with an actual issue.