search

telepresenceio / telepresence

Local development against a remote Kubernetes or OpenShift cluster
https://www.telepresence.io
Other
6.53k stars 513 forks source link

Timeout Error While Attempting to Intercept Rollout Service Using Telepresence #3691

Open simorla opened 13 hours ago

simorla commented 13 hours ago

We have created a rollout service and are trying to intercept it using Telepresence. However, we're encountering a timeout error.

Command:

telepresence --namespace ping-services intercept microservice-generic-smorla-69dd7655dd --service microservice-generic-smorla --port 8080:80 --mount=false --env-json /Users/smorla/microservice-generic/env.json

Error:

W0930 18:17:02.561188 8236 native_arm64.go:52] Could not read /proc/cpuinfo: open /proc/cpuinfo: no such file or directory W0930 18:17:02.561239 8236 native_arm64.go:177] Could not read /proc/self/auxv: open /proc/self/auxv: no such file or directory Error: request timed out while waiting for agent microservice-generic-smorla-69dd7655dd.ping-services to arrive

Service Details:

kubectl -n ping-services describe service microservice-generic-smorla

Name: microservice-generic-smorla Namespace: ping-services Labels: allow-deletion-by-users=true app.name=microservice-generic telepresence=true user.name=smorla Type: ClusterIP IP: Port: 80/TCP TargetPort: 8080/TCP Endpoints: Session Affinity: None

Relevant Pods in telepresence kubectl get pods -n ambassador

traffic-manager-595d7f4558-f89tt 1/1 Running 0 6d4h traffic-manager-ambassador-agent-df7f4ccfb-shqxj 1/1 Running 0 6d17h

kubectl get all -n ping-services | grep smorla

pod/microservice-generic-smorla-69dd7655dd-5jg56 2/2 Running 24m service/microservice-generic-smorla ClusterIP 80/TCP 52m replicaset.apps/microservice-generic-smorla-69dd7655dd 1 1 1 51m

### Telepresence Version:"

Client: v2.7.2 (api v3) Root Daemon: v2.7.2 (api v3) User Daemon: v2.7.2 (api v3)

Telepresence Status: Root Daemon: Running Version : v2.7.2 (api 3) DNS : Remote IP: 127.0.0.1 User Daemon: Running Version : v2.7.2 (api 3) Intercepts: 0 total

Timeout Settings: We have tried increasing the timeouts.agentArrival to 120s, but it hasn't resolved the issue

helm upgrade traffic-manager datawire/telepresence --namespace ambassador --set timeouts.agentArrival=120s

Any guidance or suggestions to resolve this issue would be greatly appreciated.

Logs:: 2024-09-30 17:59:20.9023 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.alloy.local 2024-09-30 17:59:20.9025 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.otel.local 2024-09-30 17:59:20.9027 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.ping-cassandra.local 2024-09-30 17:59:20.9028 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.kube-node-lease.local 2024-09-30 17:59:20.9032 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.applicationroles-crud.local 2024-09-30 17:59:20.9034 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.node-problem-detector.local 2024-09-30 17:59:20.9036 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.tag1-v2.local 2024-09-30 17:59:20.9037 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.telemetry.local 2024-09-30 17:59:20.9039 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.olm.local 2024-09-30 17:59:20.9040 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.default.local 2024-09-30 17:59:20.9042 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.branding1.local 2024-09-30 17:59:20.9044 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.monitoring.local 2024-09-30 17:59:20.9046 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.jenkins-infra-test.local 2024-09-30 17:59:20.9051 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.mock-smtp-p1-perf.local 2024-09-30 17:59:20.9060 debug daemon/session/dns/SearchPaths : Performing initial recursion check with tel2-recursion-check.kube-system 2024-09-30 17:59:20.9805 debug daemon/session/dns/Server : LookupHost "tel2-recursion-check.kube-system" 2024-09-30 17:59:20.9807 debug daemon/session/dns/Server : SVCB _dns.resolver.arpa. -> NXDOMAIN 2024-09-30 17:59:21.4116 debug daemon/session/dns/Server : DNS resolver is not recursive 2024-09-30 17:59:21.4120 debug daemon/session/dns/Server : A tel2-recursion-check.kube-system. -> SERVFAIL rpc error: code = Unimplemented desc = unknown method LookupHost for service telepresence.manager.Manager 2024-09-30 17:59:21.4190 debug daemon/session/dns/SearchPaths : Recursion check finished 2024-09-30 17:59:20.9023 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.alloy.local 2024-09-30 17:59:20.9025 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.otel.local 2024-09-30 17:59:20.9027 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.ping-cassandra.local 2024-09-30 17:59:20.9028 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.kube-node-lease.local 2024-09-30 17:59:20.9032 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.applicationroles-crud.local 2024-09-30 17:59:20.9034 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.node-problem-detector.local 2024-09-30 17:59:20.9036 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.tag1-v2.local 2024-09-30 17:59:20.9037 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.telemetry.local 2024-09-30 17:59:20.9039 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.olm.local 2024-09-30 17:59:20.9040 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.default.local 2024-09-30 17:59:20.9042 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.branding1.local 2024-09-30 17:59:20.9044 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.monitoring.local 2024-09-30 17:59:20.9046 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.jenkins-infra-test.local 2024-09-30 17:59:20.9051 info daemon/session/dns/SearchPaths : Generated new /etc/resolver/telepresence.mock-smtp-p1-perf.local 2024-09-30 17:59:20.9060 debug daemon/session/dns/SearchPaths : Performing initial recursion check with tel2-recursion-check.kube-system 2024-09-30 17:59:20.9805 debug daemon/session/dns/Server : LookupHost "tel2-recursion-check.kube-system" 2024-09-30 17:59:20.9807 debug daemon/session/dns/Server : SVCB _dns.resolver.arpa. -> NXDOMAIN 2024-09-30 17:59:21.4116 debug daemon/session/dns/Server : DNS resolver is not recursive 2024-09-30 17:59:21.4120 debug daemon/session/dns/Server : A tel2-recursion-check.kube-system. -> SERVFAIL rpc error: code = Unimplemented desc = unknown method LookupHost for service telepresence.manager.Manager 2024-09-30 17:59:21.4190 debug daemon/session/dns/SearchPaths : Recursion check finished smorla@mac-YM2KD29M test-kasparov %

cindymullins-dw commented 10 hours ago

Hi @simorla , the latest version is 2.19.6. Could you please upgrade and let us know if the issue persists?

If it does, on the latest version run ‘telepresence gather logs’. Logs in later versions address this type of error so should be more informative. Also please note the daemon logs are not very relevant here. The Traffic Manager pod logs would be more helpful.