search

TheNewNormal / kube-solo-osx

Local development Kubernetes Solo Cluster for macOS made very simple
Apache License 2.0
575 stars 29 forks source link

install hangs on '~/kube-solo/bin/kubectl get nodes' #140

Closed seanknox closed 8 years ago

seanknox commented 8 years ago

Fresh install of 0.9.6 on macOS Sierra hangs at the ~/kube-solo/bin/kubectl get nodes step:

/Applications/Kube-Solo.app/Contents/Resources/up.command; exit;
[seanknox:~]$ /Applications/Kube-Solo.app/Contents/Resources/up.command; exit;

Starting k8solo-01 VM ...

Data disk does not exist, it will be created now ...

Please type Data disk size in GBs followed by [ENTER]:
[default is 30]:

Creating 30GB sparse disk (QCow2)...
-
Created 30GB Data disk

Starting VM ...

> booting k8solo-01 (1/1)
[WARN] NFS is qnot supported by running 'corectld'
---> 'k8solo-01' started successfuly with address 192.168.64.24 and PID 7441
---> 'k8solo-01' boot logs can be found at '/Users/seanknox/.coreos/running/90394897-5956-4F9D-AFC4-E274C45DAB31/log'
---> 'k8solo-01' console can be found at '/Users/seanknox/.coreos/running/90394897-5956-4F9D-AFC4-E274C45DAB31/tty'

Checking internet availablity on VM...
Yes, internet is available ...

Installing Kubernetes files on to VM...
2016-10-04 21:13:08.608455 I | uploading 'kube.tgz' to 'k8solo-01:/home/core/kube.tgz'
101.29 MB / 101.29 MB [==============================================] 100.00 %
                                                                                Files copied to VM...
Installing now ...
Done...

Waiting for etcd service to be ready on VM...
...

Waiting for Kubernetes cluster to be ready. This can take a bit...
|...

Waiting for Kubernetes node to be ready. This can take a bit...
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x8d1415491e pc=0xebeb]

runtime stack:
runtime.throw(0x26389e0, 0x2a)
    /usr/local/go/src/runtime/panic.go:547 +0x90
runtime.sigpanic()
    /usr/local/go/src/runtime/sigpanic_unix.go:12 +0x5a
runtime.unlock(0x3391ec0)
    /usr/local/go/src/runtime/lock_sema.go:107 +0x14b
runtime.(*mheap).alloc_m(0x3391ec0, 0x1, 0x12, 0x39ce580)
    /usr/local/go/src/runtime/mheap.go:492 +0x314
runtime.(*mheap).alloc.func1()
    /usr/local/go/src/runtime/mheap.go:502 +0x41
runtime.systemstack(0x7fff5fbff088)
    /usr/local/go/src/runtime/asm_amd64.s:307 +0xab
runtime.(*mheap).alloc(0x3391ec0, 0x1, 0x10000000012, 0xe88f)
    /usr/local/go/src/runtime/mheap.go:503 +0x63
runtime.(*mcentral).grow(0x3393970, 0x0)
    /usr/local/go/src/runtime/mcentral.go:209 +0x93
runtime.(*mcentral).cacheSpan(0x3393970, 0xc8200191f8)
    /usr/local/go/src/runtime/mcentral.go:89 +0x47d
runtime.(*mcache).refill(0x39c8000, 0x12, 0xc8200f1b50)
    /usr/local/go/src/runtime/mcache.go:119 +0xcc
runtime.mallocgc.func2()
    /usr/local/go/src/runtime/malloc.go:642 +0x2b
runtime.systemstack(0x3389100)
    /usr/local/go/src/runtime/asm_amd64.s:291 +0x79
runtime.mstart()
    /usr/local/go/src/runtime/proc.go:1051

goroutine 1 [running, locked to thread]:
runtime.systemstack_switch()
    /usr/local/go/src/runtime/asm_amd64.s:245 fp=0xc8200f1778 sp=0xc8200f1770
runtime.mallocgc(0x120, 0x1afd860, 0xc800000000, 0xc8200e3ad0)
    /usr/local/go/src/runtime/malloc.go:643 +0x869 fp=0xc8200f1850 sp=0xc8200f1778
runtime.newarray(0x1afd860, 0xb, 0xc8200e3ad0)
    /usr/local/go/src/runtime/malloc.go:798 +0xc9 fp=0xc8200f1890 sp=0xc8200f1850
runtime.makeslice(0x1afb340, 0xb, 0xb, 0x0, 0x0, 0x0)
    /usr/local/go/src/runtime/slice.go:32 +0x165 fp=0xc8200f18e0 sp=0xc8200f1890
regexp.makeOnePass(0xc8200e3a10, 0xc8200e3a10)
    /usr/local/go/src/regexp/onepass.go:313 +0x2cb fp=0xc8200f1a68 sp=0xc8200f18e0
regexp.compileOnePass(0xc8200e39b0, 0xc8200e39b0)
    /usr/local/go/src/regexp/onepass.go:507 +0x1c7 fp=0xc8200f1ae8 sp=0xc8200f1a68
regexp.compile(0x25d6260, 0x2a, 0x30000d4, 0x0, 0x0, 0x0)
    /usr/local/go/src/regexp/regexp.go:179 +0x15a fp=0xc8200f1ba8 sp=0xc8200f1ae8
regexp.Compile(0x25d6260, 0x2a, 0x3a1c0a8, 0x0, 0x0)
    /usr/local/go/src/regexp/regexp.go:129 +0x43 fp=0xc8200f1be0 sp=0xc8200f1ba8
regexp.MustCompile(0x25d6260, 0x2a, 0x5bb79b)
    /usr/local/go/src/regexp/regexp.go:230 +0x3d fp=0xc8200f1c80 sp=0xc8200f1be0
k8s.io/kubernetes/pkg/util/validation.init()
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/validation/validation.go:32 +0x74 fp=0xc8200f1ca0 sp=0xc8200f1c80
k8s.io/kubernetes/pkg/labels.init()
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/labels/selector.go:822 +0x65 fp=0xc8200f1cd8 sp=0xc8200f1ca0
k8s.io/kubernetes/pkg/api/unversioned.init()
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/api/unversioned/zz_generated.deepcopy.go:390 +0x6f fp=0xc8200f1d20 sp=0xc8200f1cd8
k8s.io/kubernetes/pkg/api.init()
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/api/zz_generated.deepcopy.go:3749 +0x6e fp=0xc8200f1ea0 sp=0xc8200f1d20
k8s.io/kubernetes/pkg/kubectl/cmd.init()
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/version.go:55 +0x75 fp=0xc8200f1f30 sp=0xc8200f1ea0
k8s.io/kubernetes/cmd/kubectl/app.init()
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/app/kubectl.go:39 +0x45 fp=0xc8200f1f38 sp=0xc8200f1f30
main.init()
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/kubectl.go:30 +0x40 fp=0xc8200f1f40 sp=0xc8200f1f38
runtime.main()
    /usr/local/go/src/runtime/proc.go:177 +0x27f fp=0xc8200f1f90 sp=0xc8200f1f40
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc8200f1f98 sp=0xc8200f1f90

goroutine 9 [chan receive]:
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).flushDaemon(0x3388d00)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:879 +0x67
created by k8s.io/kubernetes/vendor/github.com/golang/glog.init.1
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:410 +0x297
-Unable to connect to the server: dial tcp 192.168.64.24:8080: i/o timeout
\

In another shell:

[seanknox:~]$ ps ax |grep kube
 7441   ??  S      0:34.62 /Users/seanknox/bin/corectld.runner -s 0:0,hostbridge -l com1,autopty=/Users/seanknox/.coreos/running/90394897-5956-4F9D-AFC4-E274C45DAB31/tty,log=/Users/seanknox/.coreos/running/90394897-5956-4F9D-AFC4-E274C45DAB31/log -s 5,virtio-rnd -s 31,lpc -U 90394897-5956-4F9D-AFC4-E274C45DAB31 -m 3072M -c 2 -A -u -s 2:0,virtio-net -s 4:0,virtio-blk,file:///Users/seanknox/kube-solo/data.img,format=qcow -f kexec,/Users/seanknox/.coreos/images/stable/1122.2.0/coreos_production_pxe.vmlinuz,/Users/seanknox/.coreos/images/stable/1122.2.0/coreos_production_pxe_image.cpio.gz,earlyprintk=serial console=ttyS0 coreos.autologin coreos.first_boot=1 corectl.hostname=k8solo-01  coreos.config.url=http://192.168.64.1:2511/90394897-5956-4F9D-AFC4-E274C45DAB31/ignition cloud-config-url=http://192.168.64.1:2511/90394897-5956-4F9D-AFC4-E274C45DAB31/cloud-config
 8894 s001  R+     0:23.19 /Users/seanknox/kube-solo/bin/kubectl get nodes
rimusz commented 8 years ago

hey @seanknox this problem has nothing to do with kube-solo or kube-cluster apps, the problem is with kubectl which is compiled with go v1.6.x, on Sierra go binaries compiled with that go version spit those errors. Only go binaries compiled with go v1.7.x work fine there. https://medium.com/@gerred/kubectl-1-3-7-for-macos-sierra-ee2c483e283d#.2x4fuh2sj

seanknox commented 8 years ago

hey @rimusz! thanks for pointing me to that. Should I be compiling kubectl using go v1.7.x and then dropping it into ~/kube-solo/bin? And if so, should I do the same with https://github.com/TheNewNormal/kube-cluster-osx?

rimusz commented 8 years ago

@seanknox yes, that should solve the kubectl problem till we get k8s binaries built with go v1.7. I was not able to compile kubectl with go v1.7, as always so many go dependencies there, not really go expert.

rimusz commented 8 years ago

also I need to update deis_install there (minio), to support latest workflow move from rc to deployments

seanknox commented 8 years ago

Sweet, I did just that and all is working now. For posterity the process was:

  1. Remove all versions of go via homebrew
  2. Install go v1.7.1 via homebrew
  3. Compile kubectl according to https://medium.com/@gerred/kubectl-1-3-7-for-macos-sierra-ee2c483e283d#.l9sop4jje
  4. Stop kube-solo and rm -rf ~/kube-solo to prepare for a fresh install
  5. Start kube-solo and start the initial setup
  6. At the first prompt asking for CoreOS version, copy the newly compiled kubectl to ~/kube-solo/bin`
  7. Resume kube-solo install

Thanks for your help!