search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.5k stars 4.89k forks source link

VM may be unable to resolve external DNS records #5891

Closed riverhxz closed 5 years ago

riverhxz commented 5 years ago

start an minikube cluster on mac:

minikube start

😄  Darwin 10.13.6 上的 minikube v1.5.2
✨  Automatically selected the 'hyperkit' driver (alternates: [virtualbox vmwarefusion])
🔥  正在创建 hyperkit 虚拟机（CPUs=2，Memory=2000MB, Disk=20000MB）...
⚠️  VM may be unable to resolve external DNS records
⚠️  VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository
🐳  正在 Docker '18.09.9' 中准备 Kubernetes v1.16.2…
🚜  拉取镜像 ...
❌  无法拉取映像，有可能是正常状况：running cmd: "/bin/bash -c \"sudo env PATH=/var/lib/minikube/binaries/v1.16.2:$PATH kubeadm config images pull --config /var/tmp/minikube/kubeadm.yaml\"": Process exited with status 1
🚀  正在启动 Kubernetes ...
⌛  Waiting for: apiserver
🏄  完成！kubectl 已经配置至 "minikube"

kubectl get pods --all-namespaces

NAMESPACE     NAME                               READY   STATUS             RESTARTS   AGE
kube-system   coredns-5644d7b6d9-b64jq           1/1     Running            0          12m
kube-system   coredns-5644d7b6d9-fnb2g           1/1     Running            0          12m
kube-system   etcd-minikube                      1/1     Running            0          11m
kube-system   kube-addon-manager-minikube        0/1     ImagePullBackOff   0          12m
kube-system   kube-apiserver-minikube            1/1     Running            0          11m
kube-system   kube-controller-manager-minikube   1/1     Running            0          11m
kube-system   kube-proxy-m5npj                   1/1     Running            0          12m
kube-system   kube-scheduler-minikube            1/1     Running            0          11m

The full output of the command that failed:

In the vm. network is ok, but vm may be unable to resolve external DNS records.

$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=47 time=49.857 ms
64 bytes from 8.8.8.8: seq=1 ttl=47 time=48.944 ms
64 bytes from 8.8.8.8: seq=2 ttl=47 time=49.087 ms

$ nslookup www.baidu.com nslookup: read: Connection

$ nslookup www.google.com nslookup: read: Connection refused

also cannot pull images $ docker pull k8s.gcr.io/kube-addon-manager:v9.0.2

Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.8:38581->192.168.64.1:53: read: connection refused

Please advise.

riverhxz commented 5 years ago

use virtualbox solve the problem.

spursy commented 4 years ago

I use virtualbox, I encounter the same issue

  ~ minikube start --docker-env HTTP_PROXY=$http_proxy --docker-env HTTPA_PROXY=$https_proxy --docker-env NO_PROXY=$NO_PROXY --vm-driver=virtualbox
😄  minikube v1.5.2 on Darwin 10.14.6
🔥  Creating virtualbox VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
🌐  Found network options:
    ▪ NO_PROXY=localhost,127.0.0.1,10.96.0.0/12,192.168.99.0/24,192.168.39.0/24
    ▪ http_proxy=http://127.0.0.1:7890
    ▪ https_proxy=http://127.0.0.1:7890
⚠️  VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository
🐳  Preparing Kubernetes v1.16.2 on Docker '18.09.9' ...
    ▪ env HTTP_PROXY=http://127.0.0.1:7890
    ▪ env HTTPA_PROXY=http://127.0.0.1:7890
    ▪ env NO_PROXY=localhost,127.0.0.1,10.96.0.0/12,192.168.99.0/24,192.168.39.0/24
🚜  Pulling images ...
❌  Unable to pull images, which may be OK: running cmd: "/bin/bash -c \"sudo env PATH=/var/lib/minikube/binaries/v1.16.2:$PATH kubeadm config images pull --config /var/tmp/minikube/kubeadm.yaml\"": Process exited with status 1
🚀  Launching Kubernetes ...
⌛  Waiting for: apiserver
🏄  Done! kubectl is now configured to use "minikube"
OMGerCoder commented 4 years ago

thx for the fix

clklachu commented 4 years ago

thx for the fix

What is the fix?

ashirbadomm commented 4 years ago

Exactly, am facing the same issue using VirtualBox, no resolution available yet:

🙄 minikube v1.6.1 on Centos 7.7.1908 ✨ Selecting 'virtualbox' driver from user configuration (alternates: [kvm2 none]) 🔥 Creating virtualbox VM (CPUs=2, Memory=4096MB, Disk=20000MB) ... ⚠️ VM may be unable to resolve external DNS records ⚠️ VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository 🐳 Preparing Kubernetes v1.17.0 on Docker '19.03.5' ... 🚜 Pulling images ... ❌ Unable to pull images, which may be OK: running cmd: "/bin/bash -c \"sudo env PATH=/var/lib/minikube/binaries/v1.17.0:$PATH kubeadm config images pull --config /var/tmp/minikube/kubeadm.yaml\"": /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.17.0:$PATH kubeadm config images pull --config /var/tmp/minikube/kubeadm.yaml": Process exited with status 1 stdout:

stderr: W1230 04:29:42.501316 3417 common.go:77] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta1". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version. W1230 04:29:42.502479 3417 common.go:77] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta1". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version. W1230 04:29:42.505174 3417 validation.go:28] Cannot validate kube-proxy config - no validator is available W1230 04:29:42.505263 3417 validation.go:28] Cannot validate kubelet config - no validator is available failed to pull image "k8s.gcr.io/kube-apiserver:v1.17.0": output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 10.0.2.3:53: read udp 10.0.2.15:44940->10.0.2.3:53: i/o timeout , error: exit status 1 To see the stack trace of this error execute with --v=5 or higher

🚀 Launching Kubernetes ... ⌛ Waiting for cluster to come online ... 🏄 Done! kubectl is now configured to use "minikube"

Any resolution will be highly appreciated.

shwetankverma commented 4 years ago

I am also facing the same issue. I have docker desktop enabled and running. Any resolution please

billcrook commented 4 years ago

I'm seeing a similar issue on arch-linux using kvm2. After starting minikube and running:

minikube ssh 'docker pull alpine'

the minikube system cannot perform dns queries:

Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.122.1:53: server misbehaving

FTR, created #6229

githubcdr commented 4 years ago

Got hit by this issue too, I got it fixed by removing the bridge100 interface and restarting my machine.

The interface will be recreated and the DNS resolver will be configured again to serve on this interface. On Mac Catalina you can d;

sudo ifconfig bridge100 destroy

Only do this if you know what you're doing ;)

FilBot3 commented 4 years ago

This is the error that I get using minikube-1.6.2-windows-amd64.exe on hyperv. 

λ  Get-Date; .\Downloads\minikube-1.6.2-windows-amd64.exe start --vm-driver=hyperv; Get-Date

Friday, March 6, 2020 2:09:32 PM
* minikube 1.7.3 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.7.3
* To disable this notice, run: 'minikube config set WantUpdateNotification false'

! minikube v1.6.2 on Microsoft Windows 10 Enterprise 10.0.17763 Build 17763
* Selecting 'hyperv' driver from user configuration (alternates: [])
* Downloading VM boot image ...
    > minikube-v1.6.0.iso.sha256: 65 B / 65 B [--------------] 100.00% ? p/s 0s
    > minikube-v1.6.0.iso: 150.93 MiB / 150.93 MiB [-] 100.00% 5.73 MiB p/s 27s
* Creating hyperv VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
! VM may be unable to resolve external DNS records
! VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository
* Preparing Kubernetes v1.17.0 on Docker '19.03.5' ...
* Downloading kubeadm v1.17.0
* Downloading kubelet v1.17.0
* Pulling images ...
* Unable to pull images, which may be OK: running cmd: "/bin/bash -c \"sudo env PATH=/var/lib/minikube/binaries/v1.17.0:$PATH kubeadm config images pull --config /var/tmp/minikube/kubeadm.yaml\"": /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.17.0:$PATH kubeadm config images pull --config /var/tmp/minikube/kubeadm.yaml": Process exited with status 1
stdout:

stderr:
W0306 20:14:58.254185    3455 common.go:77] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta1". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W0306 20:14:58.255061    3455 common.go:77] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta1". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W0306 20:14:58.257081    3455 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0306 20:14:58.257128    3455 validation.go:28] Cannot validate kubelet config - no validator is available
failed to pull image "k8s.gcr.io/kube-apiserver:v1.17.0": output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on [2001:4860:4860::8888]:53: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
, error: exit status 1
To see the stack trace of this error execute with --v=5 or higher

* Launching Kubernetes ...
*
X Error starting cluster: init failed. cmd: "/bin/bash -c \"sudo env PATH=/var/lib/minikube/binaries/v1.17.0:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap\"": /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.17.0:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap": Process exited with status 1
stdout:

stderr:
W0306 20:14:58.386188    3469 common.go:77] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta1". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W0306 20:14:58.388259    3469 common.go:77] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta1". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W0306 20:14:58.389197    3469 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0306 20:14:58.389229    3469 validation.go:28] Cannot validate kubelet config - no validator is available
cannot use "fe80::215:5dff:fe00:290e" as the bind address for the API Server
To see the stack trace of this error execute with --v=5 or higher

*
* minikube is exiting due to an error. If the above message is not useful, open an issue:
  - https://github.com/kubernetes/minikube/issues/new/choose
Friday, March 6, 2020 2:15:00 PM

I chose to use that version as it was the last stable version before 1.7.* series and the 1.7.3 version didn't actually seem to adhere to my timeout modifications or show any kind of verbosity in the command when given --v=7.

Then I use the --image-repository= statement like mentioned.

C:\Users\dudleyp
λ  ping k8s.gcr.io

Pinging googlecode.l.googleusercontent.com [108.177.103.82] with 32 bytes of data:
Reply from 108.177.103.82: bytes=32 time=28ms TTL=39
Reply from 108.177.103.82: bytes=32 time=28ms TTL=39
Reply from 108.177.103.82: bytes=32 time=30ms TTL=39
Reply from 108.177.103.82: bytes=32 time=28ms TTL=39

Ping statistics for 108.177.103.82:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 28ms, Maximum = 30ms, Average = 28ms
C:\Users\dudleyp
λ  Get-Date; .\Downloads\minikube-1.6.2-windows-amd64.exe start --vm-driver=hyperv --image-repository=108.177.103.82; Get-Date

Friday, March 6, 2020 2:56:20 PM
* minikube v1.6.2 on Microsoft Windows 10 Enterprise 10.0.17763 Build 17763
* Selecting 'hyperv' driver from user configuration (alternates: [])
* Using image repository 108.177.103.82
* Creating hyperv VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x28 pc=0x12ee976]

goroutine 89 [running]:
github.com/google/go-containerregistry/pkg/v1/tarball.Write(0x0, 0xc0004d7890, 0xe, 0xc0004d789f, 0x17, 0xc0004d78b7, 0x7, 0x0, 0x0, 0xc00065dc58, ...)
        /go/pkg/mod/github.com/google/go-containerregistry@v0.0.0-20180731221751-697ee0b3d46e/pkg/v1/tarball/write.go:57 +0x136
k8s.io/minikube/pkg/minikube/machine.CacheImage(0xc0004d7890, 0x2e, 0xc0006ae2a0, 0x56, 0x0, 0x0)
        /app/pkg/minikube/machine/cache_images.go:395 +0x615
k8s.io/minikube/pkg/minikube/machine.CacheImages.func1(0xc00028ff68, 0x3)
        /app/pkg/minikube/machine/cache_images.go:85 +0xed
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc000640180, 0xc000640210)
        /go/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:57 +0x6b
created by golang.org/x/sync/errgroup.(*Group).Go
        /go/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:54 +0x6d
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x28 pc=0x12ee976]

goroutine 96 [running]:
github.com/google/go-containerregistry/pkg/v1/tarball.Write(0x0, 0xc0004d7950, 0xe, 0xc0004d795f, 0x9, 0xc0004d7969, 0xc, 0x0, 0x0, 0xc000763c58, ...)
        /go/pkg/mod/github.com/google/go-containerregistry@v0.0.0-20180731221751-697ee0b3d46e/pkg/v1/tarball/write.go:57 +0x136
k8s.io/minikube/pkg/minikube/machine.CacheImage(0xc0004d7950, 0x25, 0xc0006176d0, 0x4d, 0x0, 0x0)
        /app/pkg/minikube/machine/cache_images.go:395 +0x615
k8s.io/minikube/pkg/minikube/machine.CacheImages.func1(0xc00069ff68, 0x0)
        /app/pkg/minikube/machine/cache_images.go:85 +0xed
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc000640180, 0xc000640360)
        /go/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:57 +0x6b
created by golang.org/x/sync/errgroup.(*Group).Go
        /go/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:54 +0x6d
Friday, March 6, 2020 2:56:23 PM
Trupal00p commented 4 years ago

For others who come accross this, here is what fixed this for me:

I have docker desktop installed on a Windows 10 machine alongside minikube. I had turned off the docker desktop service. when I turned it back on and restarted the minikube vm (minkube stop && minikube start) everything started working again.

I think docker desktop may be doing something with DNS lookups but I'm not sure how or where it might be doing this.