telepresenceio / telepresence

Local development against a remote Kubernetes or OpenShift cluster
https://www.telepresence.io
Other
6.52k stars 513 forks source link

Custom Timeout for wait_for_pod #1393

Closed CoolDarran closed 2 weeks ago

CoolDarran commented 4 years ago

The default timeout waiting for pod is fixed. It will be nice if we can set a custom timeout in command. https://github.com/telepresenceio/telepresence/blob/ed4f9ad54a765bf3019c4517676052dad9c9be7d/telepresence/proxy/remote.py#L105

In my situation, the pod is under creation, but crashed after 120s

 128.4 TEL | [142] captured in 0.34 secs.
 128.4 TEL | END SPAN remote.py:104(wait_for_pod)  120.2s
 130.4 TEL | CRASH: Pod isn't starting or can't be found: {'conditions': [{'lastProbeTime': None,'lastTransitionTime': '2020-06-30T09:00:03Z','message': 'containers with incomplete status: [check-service-apollo-admin-server-sit sw-agent-sidecar]','reason': 'ContainersNotInitialized','status': 'False','type': 'Initialized'}, {'lastProbeTime': None,'lastTransitionTime': '2020-06-30T09:00:03Z','message': 'containers with unready status: [idt-app-manager]','reason': 'ContainersNotReady','status': 'False','type': 'Ready'}, {'lastProbeTime': None,'lastTransitionTime': '2020-06-30T09:00:03Z','message': 'containers with unready status: [idt-app-manager]','reason': 'ContainersNotReady','status': 'False','type': 'ContainersReady'}, {'lastProbeTime': None,'lastTransitionTime': '2020-06-30T09:00:03Z','status': 'True','type': 'PodScheduled'}],'containerStatuses': [{'image': 'datawire/telepresence-k8s:0.105','imageID': '','lastState': {},'name': 'idt-app-manager','ready': False,'restartCount': 0,'started': False,'state': {'waiting': {'reason': 'PodInitializing'}}}],'hostIP': '172.18.75.249','initContainerStatuses': [{'image': 'idt/alpine-bash:3.8','imageID': '','lastState': {},'name': 'check-service-apollo-admin-server-sit','ready': False,'restartCount': 0,'state': {'waiting': {'reason': 'PodInitializing'}}}, {'image': 'idt/skywalking-agent-jvmkill:6.6.0','imageID': '','lastState': {},'name': 'sw-agent-sidecar','ready': False,'restartCount': 0,'state': {'waiting': {'reason': 'PodInitializing'}}}],'phase': 'Pending','qosClass': 'Burstable','startTime': '2020-06-30T09:00:03Z'}
 130.4 TEL | Traceback (most recent call last):
 130.4 TEL |   File "/usr/local/bin/telepresence/telepresence/cli.py", line 135, in crash_reporting
 130.4 TEL |     yield
 130.4 TEL |   File "/usr/local/bin/telepresence/telepresence/main.py", line 65, in main
 130.4 TEL |     remote_info = start_proxy(runner)
 130.4 TEL |   File "/usr/local/bin/telepresence/telepresence/proxy/__init__.py", line 134, in start_proxy
 130.4 TEL |     run_id=run_id,
 130.4 TEL |   File "/usr/local/bin/telepresence/telepresence/proxy/remote.py", line 193, in get_remote_info
 130.4 TEL |     wait_for_pod(runner, remote_info)
 130.4 TEL |   File "/usr/local/bin/telepresence/telepresence/proxy/remote.py", line 125, in wait_for_pod
 130.4 TEL |     "Pod isn't starting or can't be found: {}".format(pod["status"])
 130.4 TEL | RuntimeError: Pod isn't starting or can't be found: {'conditions': [{'lastProbeTime': None,'lastTransitionTime': '2020-06-30T09:00:03Z','message': 'containers with incomplete status: [check-service-apollo-admin-server-sit sw-agent-sidecar]','reason': 'ContainersNotInitialized','status': 'False','type': 'Initialized'}, {'lastProbeTime': None,'lastTransitionTime': '2020-06-30T09:00:03Z','message': 'containers with unready status: [idt-app-manager]','reason': 'ContainersNotReady','status': 'False','type': 'Ready'}, {'lastProbeTime': None,'lastTransitionTime': '2020-06-30T09:00:03Z','message': 'containers with unready status: [idt-app-manager]','reason': 'ContainersNotReady','status': 'False','type': 'ContainersReady'}, {'lastProbeTime': None,'lastTransitionTime': '2020-06-30T09:00:03Z','status': 'True','type': 'PodScheduled'}],'containerStatuses': [{'image': 'datawire/telepresence-k8s:0.105','imageID': '','lastState': {},'name': 'idt-app-manager','ready': False,'restartCount': 0,'started': False,'state': {'waiting': {'reason': 'PodInitializing'}}}],'hostIP': '172.18.75.249','initContainerStatuses': [{'image': 'idt/alpine-bash:3.8','imageID': '','lastState': {},'name': 'check-service-apollo-admin-server-sit','ready': False,'restartCount': 0,'state': {'waiting': {'reason': 'PodInitializing'}}}, {'image': 'idt/skywalking-agent-jvmkill:6.6.0','imageID': '','lastState': {},'name': 'sw-agent-sidecar','ready': False,'restartCount': 0,'state': {'waiting': {'reason': 'PodInitializing'}}}],'phase': 'Pending','qosClass': 'Burstable','startTime': '2020-06-30T09:00:03Z'}
 130.4 TEL | (calling crash reporter...)
github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment, or this will be closed in 7 days.

github-actions[bot] commented 2 weeks ago

This issue was closed because it has been stalled for 7 days with no activity.