Closed pquentin closed 7 years ago
mboersma
commented
8 years ago I'm having the same problem described above, as I've noted in #646. But I've tried both brew install docker and brew cask install docker and neither one works well with minikube + xhyve on my Mac.
dlorenc
commented
8 years ago @zchee do you think this is the same UUID issue you've mentioned a few other times?
0ff
commented
8 years ago With xhyve + docker for mac, I can ping 8.8.8.8 and google.com successfully, but I can't curl google.com: it timeouts (separate issue but I thought I should mention it).
@pquentin The exact same thing happens for me - for some reason, minikube decides to use the host as the DNS Server in this setup. See the /etc/resolv.conf file in your minikube instance (i.e. minikube ssh); if you change this (or run dnsmasq on your host and bind it to your minikube's ip) it'll work just fine.
pquentin
commented
8 years ago @0ff Changed it to 8.8.8.8 but there's no change. I don't think it was using the host ip on my network though. What's surprising too is that I can ping google.com, and this requires DNS resolution, right?
Hmm, so I just managed to create a minikube with xhyve + docker (not for mac). I think the only thing I did differently was to remove ~/.minikube. This appears to confirm what @mboersma said, Docker for Mac does not seem to actually change things here, so I'll remove that from the title.
zchee
commented
8 years ago @dlorenc maybe yes. I'll detail commnet later.
but short answer:
vmnet write some macOS internal files and convert hardcoded UUIDkill -9 to xhyve-driver(just example), but not cleanup internal files
jimmidyson
commented
8 years ago The reason for the hardcoded UUID was to try to get a consistent IP across restarts. Do you think that is still going to be possible?
zchee
commented
8 years ago @jimmidyson Yeah, I know. But vmnet interface is not simply. actually, I had same issue at previous. So now test and use disabled use hardcoded UUID version.(means, self build minikube binary) maybe, it solved remove /var/db/dhcpd_leases and /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist, but very risky for end-users.
Now I consider and research solution...
zchee
commented
8 years ago And just question. How did you decide UUID? use uuidgen?
jimmidyson
commented
8 years ago I didn't... @chirino did iirc.
zchee
commented
8 years ago @jimmidyson @dlorenc It might not relate, but macOS have original(?) UUID rules. (I think and maybe)
When I use satori/go.uuid(or etc. google/uuid is same) for generate UUID, sometimes test is fail.
It seems that according to the RFC 4122, but vmnet.framework has recognized it as invalid.
This is why I have to generate a uuid using cgo. https://github.com/zchee/docker-machine-driver-xhyve/blob/master/xhyve/uuid.go
I think control of the UUID is should carefully. But, as https://github.com/zchee/docker-machine-driver-xhyve#does-not-clean-up-the-vmnet-when-remove-a-vm, I don't yet understand vmnet.framework internal.
If we want to the same IP with no bugs, needs cleanup /var/db/dhcpd_leases and /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist(or etc). and should be check valid(duplicate?) UUID process.
Edit: uuidgen command line source Sierra version:
http://opensource.apple.com/source/xnu/xnu-3248.60.10/libkern/uuid/uuid.c
jimmidyson
commented
8 years ago @zchee You're far more knowledgeable than me on this! I'm not even a mac user... but I'm really grateful for your investigation as it's affecting a number of users. I'm sure we can find quite a few testers of any fixes you propose.
chirino
commented
8 years ago Note that in the original issue reported is not using the static UUID. Logging is different when the static UUID is used.
r2d4(minikube) Generate UUID...
(minikube) DBG | Generated UUID: 99C7DD29-AAD5-4633-9E15-4DE3F8BBCC97when the static UUID is used, you will see this in the logs instead.
(minikube) Using Supplied UUID: 57FD2012-FA4A-4FF7-AEFF-26E1A1D76847@jimmidyson oh, thanks :) I'll do my best.
@chirino Yes, and other issue thread have same bug. Now I research it.
I'll create repository that sample and simple UUID convert script written Go later.
zchee
commented
8 years ago @r2d4 Got it. thanks.
maybe caused by vmnet internal process or conver(search) IP from dhcpd_leases. I'll debug it.
chirino
commented
8 years ago FYI i've run into a similar issue in the past. What I did wrong was trying to build docker-machine-driver-xhyve using plain go get. Builds fine but it does not work. You HAVE to build with make.
zchee
commented
8 years ago @chirino Ah, good information. I'll also check the go get plain build binary behavior.
zchee
commented
8 years ago @pquentin and small reply
With xhyve + docker for mac, I can ping 8.8.8.8 and google.com successfully, but I can't curl google.com: it timeouts (separate issue but I thought I should mention it).
It means use Docker for Mac only? If so, It might solve with
System Preferences -> Network -> Location -> Edit Location... -> +(create clean setting)
It might is associated as well with this problem.
pquentin
commented
8 years ago @zchee Thanks! It does not appear to help though. I hope I did that correctly.
As I was saying, it turns out that there's no longer a difference between Docker for Mac and Docker. Maybe because I now remove ~/.minikube each time. (I also hope that it was really Docker and Docker for Mac above, because docker -v now returns something different for the two versions. I can produce the logs again if that would help.)
The minikube creation itself is no longer a problem for me: I would be happy to close this issue, but it looks like it's now bigger than me. :)
mboersma
commented
8 years ago You HAVE to build [docker-machine-driver-xhyve] with make
I've been doing this as @zchee suggested.
I would be happy to close this issue, but it looks like it's now bigger than me.
Unfortunately, yes, at least for me and a couple coworkers. Using xhyve and minikube v0.12.0 ends up here:
$ minikube start --logtostderr --show-libmachine-logs --vm-driver=xhyve
I1024 14:19:39.583187 22679 notify.go:111] Checking for updates...
Starting local Kubernetes cluster...
I1024 14:19:39.816177 22679 cluster.go:79] Machine exists!
I1024 14:19:39.836294 22679 cluster.go:86] Machine state: Running
Waiting for SSH to be available...
E1024 14:22:39.968309 22679 start.go:90] Error starting host: Error configuring auth on host: %s: Temporary Error: Too many retries waiting for SSH to be available. Last error: Maximum number of retries (60) exceeded. Retrying.
E1024 14:22:39.968418 22679 start.go:96] Error starting host: Error configuring auth on host: %s: Temporary Error: Too many retries waiting for SSH to be available. Last error: Maximum number of retries (60) exceededI've also tried with and without the "regenerate UUID" patch to the xhyve driver, but it doesn't seem to matter.
zchee
commented
8 years ago @mboersma Sorry for the delay.
I was some debug and research vmnet.framework internal. I understand some of the internal implementation. but still not understand causes.
However, found a good debugging way.
AFAIK, If you already disable SIP macOS feature(because not handled some error if enable), you can get a useful debug log in it.
open -a Console.appand re-try minikube start .... and visual-grep(check with your eyes) to Console.app log...
If you are confused by many of the logs, add to search dialog.
subsystem: com.apple.networkPlease try it and tell me the error log.
In my case, I was tested case of have not sudo privilege with enabling virtio-net. (use original xhyve command, not use docker-machine-driver-xhyve and minikube)
Ultimately, I found that internet sharing was processed in the
xhyve -A -U 87ADA66A-775D-4070-8DEA-B83AB3A47C60 -l com1,stdio -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -f 'kexec,/path/to/vmlinuz,/path/to/initrd,"earlyprintk=serial console=ttyS0 bootmem_debug debug loglevel=7'
|
/System/Library/Frameworks/vmnet.framework
|
/System/Library/PrivateFrameworks/Netrb.framework
bhundven
commented
8 years ago I have the same output as @mboersma.
adamreese
commented
7 years ago Here is my console log https://gist.github.com/adamreese/a82c10b39b438de7328cbc7f475aa61f
zchee
commented
7 years ago @adamreese Thanks for detail log!
It seems failed nw_interface_create_with_name in the libsystem_network.dylib.
I'll debug it and read https://opensource.apple.com/.
puja108
commented
7 years ago Like I already mentioned in https://github.com/kubernetes/minikube/issues/646 this is now solved in my system with some latest updates. Not sure which one actually fixed it, but my current versions are:
minikube version: v0.16.0
docker-machine version 0.9.0, build 15fd4c7
docker-machine-driver-xhyve: stable 0.3.1 (bottled), HEAD
xhyve: stable 0.2.0 (bottled), HEAD```
andyp1per
commented
7 years ago It's not working for me. In Console.app I see:
docker-machine-driver-xhyve[982] invalid client error: aborting XPC connection to daemon interface 0x7000600, netrb client create failed
in the minikube log:
(minikube) Convert UUID to MAC address... E0515 14:12:41.673413 1746 start.go:119] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Could not convert the UUID to MAC address: exit status 1.
maduramarathe9
commented
7 years ago Hi,
I followed these steps:
minikube failed to start
" $ minikube start --vm-driver=xhyve Starting local Kubernetes v1.7.0 cluster... Starting VM... Downloading Minikube ISO 97.80 MB / 97.80 MB [==============================================] 100.00% 0s E0822 23:28:00.726776 57157 start.go:129] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Could not convert the UUID to MAC address: exit status 1.
Please enter your response [Y/n]: y
"
Please let me know where I am going wrong.
maduramarathe9
commented
7 years ago rm -rf ~/.minikube sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
minikube start --vm-driver=xhyve
This helped resolve my issue:
$ minikube start --vm-driver=xhyve Starting local Kubernetes v1.7.0 cluster... Starting VM... Downloading Minikube ISO 97.80 MB / 97.80 MB [==============================================] 100.00% 0s Getting VM IP address... Moving files into cluster... Setting up certs... Starting cluster components... Connecting to cluster... Setting up kubeconfig... Kubectl is now configured to use the cluster.
pkbelidha
commented
7 years ago $kubectl run hello-minikube --image=grc.io/google_container/echoserver:1.4 --port=8080
Error: failed to discover supported resources: Get https://192.168.99.100:8443/api: dial tcp 192.168.99.100:8443: i/o timeout
Can you please help me how to resolve this ?
ankurkumarz
commented
7 years ago Yes it worked for me as well.
Please follow the same order.
sudo rm -rf ~/.minikube sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve minikube start --vm-driver=xhyve
joeljohnston
commented
6 years ago I'm having this same problem with both minikube and docker-machine using the docker-machine-driver-xhyve 0.33 release. The problem began happening after i upgraded from minikube 0.25.0 to minikube 0.25.2. The solution above does not resolve the issue. I've emptied the dhcpd_leases file and ensured permissions are set appropriately to no avail. Please help. Thanks.
Minikube version (use
minikube version): 0.11.0Environment:
cat ~/.minikube/machines/minikube/config.json | grep DriverName): xhyvedocker -v): Docker version 1.12.2, build bb80604 installed through homebrewWhat happened:
My team uses Kubernetes in production and we recently decided to switch local development from docker-compose to minikube. Their migration went well, but not mine. I tried to create a minikube cluster like this:
As you can see from the shortened logs, this does not work.
What you expected to happen:
I expected cluster creation to work. What's strange is that it does work after installing Docker for Mac using
brew cask install docker(and notbrew install docker). But it's the same Docker version!(I think it would have worked with Docker Toolbox too.)
How to reproduce it (as minimally and precisely as possible):
Update: While the steps were completely reproducible for me, they no longer are, see https://github.com/kubernetes/minikube/issues/708#issuecomment-254779233
Anything else do we need to know:
curl google.com: it timeouts (separate issue but I thought I should mention it).