search

telepresenceio / telepresence

Local development against a remote Kubernetes or OpenShift cluster
https://www.telepresence.io
Other
6.58k stars 519 forks source link

No internet access with native Mac Kubernetes (docker-for-desktop) #736

Closed soujiro32167 closed 5 years ago

soujiro32167 commented 6 years ago

Description

After launching telepresence (without any arguments), domains from Kubernetes are accessible to Chrome, but nothing outside of Kubernetes.

Otherwise, containers can see the internet:

➜  ~ k exec debugger-9bc78cc87-ngmfq -- ping github.com
PING github.com (192.30.253.113): 56 data bytes
64 bytes from 192.30.253.113: seq=42836 ttl=37 time=28.592 ms
64 bytes from 192.30.253.113: seq=1 ttl=37 time=29.497 ms

With gcloud (GKE) and telepresence, both Kubernetes and all other domains work fine

Details

Server: Engine: Version: 18.06.0-ce API version: 1.38 (minimum version 1.12) Go version: go1.10.3 Git commit: 0ffa825 Built: Wed Jul 18 19:13:46 2018 OS/Arch: linux/amd64 Experimental: false Kubernetes: Version: v1.10.3 StackAPI: v1beta2

- Telepresence version: **0.90**
- K8s version: **1.10.3**

From telepresence.log:

92.8 18 | 2018-08-14T20:54:25+0000 [stdout#info] Failed to lookup b'safebrowsing.googleapis.com' due to [Failure instance: Traceback (failure with no frames): <class 'twisted.names.error.DomainError'>: [Errno -2] Name does not resolve 92.8 18 | 2018-08-14T20:54:25+0000 [stdout#info] ], falling back to b'safebrowsing.googleapis.com.wework.com' 92.9 18 | 2018-08-14T20:54:25+0000 [-] DNSDatagramProtocol starting on 20818 92.9 18 | 2018-08-14T20:54:25+0000 [-] Starting protocol <twisted.names.dns.DNSDatagramProtocol object at 0x7fb83282fb38> 92.9 18 | 2018-08-14T20:54:25+0000 [-] (UDP Port 7796 Closed) 92.9 18 | 2018-08-14T20:54:25+0000 [-] Stopping protocol <twisted.names.dns.DNSDatagramProtocol object at 0x7fb8327ef208> 93.2 18 | 2018-08-14T20:54:25+0000 [stdout#info] getaddrinfo error: [Errno -3] Try again 93.2 18 | 2018-08-14T20:54:25+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve 95.9 18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error 95.9 18 | Traceback (most recent call last): 95.9 18 | Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)] 95.9 18 | 96.0 18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error 96.0 18 | Traceback (most recent call last): 96.0 18 | Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)] 96.0 18 | 96.0 18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error 96.0 18 | Traceback (most recent call last): 96.0 18 | Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)] 96.0 18 | 96.0 18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error 96.0 18 | Traceback (most recent call last): 96.0 18 | Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)] 96.0 18 | 96.0 18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error 96.0 18 | Traceback (most recent call last): 96.0 18 | Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)] 96.0 18 | 96.0 18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error 96.0 18 | Traceback (most recent call last): 96.0 18 | Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)] 96.0 18 | 96.0 18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error 96.0 18 | Traceback (most recent call last): 96.0 18 | Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)] 96.0 18 | 96.0 18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error 96.0 18 | Traceback (most recent call last): 96.0 18 | Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)] 96.0 18 | 96.0 18 | 2018-08-14T20:54:28+0000 [-] (UDP Port 61835 Closed) 96.0 18 | 2018-08-14T20:54:28+0000 [-] Stopping protocol <twisted.names.dns.DNSDatagramProtocol object at 0x7fb832841860> 97.7 25 | c : DNS request from ('10.65.108.0', 24811) to None: 45 bytes 97.7 18 | 2018-08-14T20:54:29+0000 [stdout#info] A query: b'safebrowsing.googleapis.com' 97.8 18 | 2018-08-14T20:54:30+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve 97.8 25 | c : DNS request from ('10.65.108.0', 55676) to None: 56 bytes 97.8 18 | 2018-08-14T20:54:30+0000 [stdout#info] Updated query of type 1 from b'safebrowsing.googleapis.com.wework.com' to b'safebrowsing.googleapis.com' 97.8 18 | 2018-08-14T20:54:30+0000 [stdout#info] A query: b'safebrowsing.googleapis.com' 97.8 18 | 2018-08-14T20:54:30+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve 97.8 18 | 2018-08-14T20:54:30+0000 [stdout#info] Failed to lookup b'safebrowsing.googleapis.com' due to [Failure instance: Traceback (failure with no frames): <class 'twisted.names.error.DomainError'>: [Errno -2] Name does not resolve 97.9 18 | 2018-08-14T20:54:30+0000 [stdout#info] ], falling back to b'safebrowsing.googleapis.com.wework.com' 100.8 25 | c : DNS request from ('10.65.108.0', 37037) to None: 27 bytes 100.8 18 | 2018-08-14T20:54:33+0000 [stdout#info] A query: b'ide.c9.io' 100.8 18 | 2018-08-14T20:54:33+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve 100.8 25 | c : DNS request from ('10.65.108.0', 55315) to None: 38 bytes 100.8 18 | 2018-08-14T20:54:33+0000 [stdout#info] Updated query of type 1 from b'ide.c9.io.wework.com' to b'ide.c9.io' 100.8 18 | 2018-08-14T20:54:33+0000 [stdout#info] A query: b'ide.c9.io' 100.8 18 | 2018-08-14T20:54:33+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve 100.8 18 | 2018-08-14T20:54:33+0000 [stdout#info] Failed to lookup b'ide.c9.io' due to [Failure instance: Traceback (failure with no frames): <class 'twisted.names.error.DomainError'>: [Errno -2] Name does not resolve 100.8 18 | 2018-08-14T20:54:33+0000 [stdout#info] ], falling back to b'ide.c9.io.wework.com' 102.8 25 | c : DNS request from ('10.65.108.0', 44676) to None: 56 bytes 102.8 18 | 2018-08-14T20:54:35+0000 [stdout#info] Updated query of type 1 from b'safebrowsing.googleapis.com.wework.com' to b'safebrowsing.googleapis.com' 102.8 18 | 2018-08-14T20:54:35+0000 [stdout#info] A query: b'safebrowsing.googleapis.com' 102.9 18 | 2018-08-14T20:54:35+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve 102.9 18 | 2018-08-14T20:54:35+0000 [stdout#info] Failed to lookup b'safebrowsing.googleapis.com' due to [Failure instance: Traceback (failure with no frames): <class 'twisted.names.error.DomainError'>: [Errno -2] Name does not resolve 102.9 18 | 2018-08-14T20:54:35+0000 [stdout#info] ], falling back to b'safebrowsing.googleapis.com.wework.com'

ark3 commented 6 years ago

Thanks for the issue. I believe this problem is due to a DNS loop. We need to update check_if_in_local_vm to catch this case.

soujiro32167 commented 6 years ago

Thanks for the super quick reply!

thedodd commented 6 years ago

I'm running into the same issue. Only when running a local process, not when using --docker-run.

thedodd commented 6 years ago

It also seems as though my process running under --docker-run can not communicate with other services. The telepresence log is showing that an A record is resolved with a specific IP, sends it back, but my process is not able to connect to it.

Could be related. Also, it is an HTTP/2 gRPC connection which is failing. Figured I would add that in as context.

vemv commented 6 years ago

This bug is obviously pretty bad - you'll agree what it's hard to develop (or work at all: think Slack, Email etc) without an internet connection.

Could I please have an approximate ETA for a fix? Not sure if it'll come reasonably soon or if contrariwise you're overwhelmed with issues.

Alternatively, is there a workaround? (which hopefully still allows one to use the VPN method, so IDEs/debuggers work)

rohansingh commented 5 years ago

The fix for this is actually very simple, I'll send a PR in a minute. In the meantime, you can temporarily work around this by running kubectl config rename-context docker-for-desktop minikube.

That will rename the context to minikube, which is handled appropriately by _check_if_in_local_vm. YMMV on how this may affect anything else you're using, though.