Closed bergwolf closed 7 years ago
On hykins, pod might take long time to start and hyperstart watcher times out. Then hyperd ignores the error and proceeds adding new containers, and eventually crashes.
I0412 08:06:22.286781 16553 hypervisor.go:54] SB[vm-CmyziupbEg] watch hyperstart, send ready: true
I0412 08:06:54.456762 16553 json.go:231] SB[vm-CmyziupbEg] got cmd:0 I0412 08:06:54.456771 16553 json.go:302] SB[vm-CmyziupbEg] send command 0 to init, payload: 'null'. I0412 08:06:54.456776 16553 json.go:231] SB[vm-CmyziupbEg] got cmd:0 I0412 08:06:54.456785 16553 json.go:302] SB[vm-CmyziupbEg] send command 0 to init, payload: 'null'. E0412 08:06:52.286973 16553 hypervisor.go:49] SB[vm-CmyziupbEg] watch hyperstart timeout I0412 08:06:54.456800 16553 json.go:88] SB[vm-CmyziupbEg] close jsonBasedHyperstart
I0412 08:06:54.473448 16553 decommission.go:554] Pod[hello121] sandbox info removed from db I0412 08:06:54.473453 16553 decommission.go:559] Pod[hello121] tag pod as stopped I0412 08:06:54.473457 16553 decommission.go:566] Pod[hello121] pod stopped time="2017-04-12T08:07:26Z" level=debug msg="container mounted via layerStore: /var/lib/hyper/rawblock/mnt/d0c4228cf2a9bbaa21497e077fa70db84b7e8d53d961a59151f64a522c9bace8/rootfs" I0412 08:07:26.345528 16553 container.go:498] Pod[hello121] Con[(hello121)] create container bcd0082849f0b07fa1ff3653377ed902805a475076343f018000d4dd2376f038 (w/: []) I0412 08:07:26.345639 16553 container.go:515] Pod[hello121] Con[bcd0082849f0(hello121)] container info config &container.Config{Hostname:"bcd0082849f0", Domainname:"", User:"", AttachStdin:false, AttachStdout:false, AttachStderr:false, ExposedPorts:map[nat.Port]struct {}(nil), PublishService:"", Tty:false, OpenStdin:false, StdinOnce:false, Env:[]string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"}, Cmd:(*strslice.StrSlice)(0xc421946c20), ArgsEscaped:false, Image:"busybox", Volumes:map[string]struct {}(nil), WorkingDir:"", Entrypoint:(*strslice.StrSlice)(nil), NetworkDisabled:true, MacAddress:"", OnBuild:[]string(nil), Labels:map[string]string{}, StopSignal:""}, Cmd [sh -c echo hello], Args [-c echo hello] I0412 08:07:26.345652 16553 container.go:520] Pod[hello121] Con[bcd0082849f0(hello121)] describe container I0412 08:07:26.345701 16553 container.go:528] Pod[hello121] Con[bcd0082849f0(hello121)] mount id: d0c4228cf2a9bbaa21497e077fa70db84b7e8d53d961a59151f64a522c9bace8 I0412 08:07:26.345748 16553 container.go:608] Pod[hello121] Con[bcd0082849f0(hello121)] Container Info is &api.ContainerDescription{Id:"bcd0082849f0b07fa1ff3653377ed902805a475076343f018000d4dd2376f038", Name:"/hello121", Image:"sha256:00f017a8c2a6e1fe2ffd05c281f27d069d2a99323a8cd514dd35f228ba26d2ff", Labels:map[string]string(nil), Tty:true, StopSignal:"TERM", RootVolume:(*api.VolumeDescription)(0xc421168460), MountId:"d0c4228cf2a9bbaa21497e077fa70db84b7e8d53d961a59151f64a522c9bace8", RootPath:"rootfs", UGI:(*api.UserGroupInfo)(nil), Envs:map[string]string{"PATH":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"}, Workdir:"/", Path:"sh", Args:[]string{"-c", "echo hello"}, Rlimits:[]*api.Rlimit{}, Sysctl:map[string]string(nil), Volumes:map[string]*api.VolumeReference(nil), Initialize:false} I0412 08:07:26.345762 16553 container.go:749] Pod[hello121] Con[bcd0082849f0(hello121)] configure dns I0412 08:07:26.345779 16553 container.go:807] Pod[hello121] Con[bcd0082849f0(hello121)] inject file /etc/resolv.conf I0412 08:07:26.717467 16553 networks.go:38] Pod[hello121] Nic[eth-default] prepare inf info: &api.InterfaceDescription{Id:"eth-default", Lo:false, Bridge:"hyper0", Ip:"192.168.123.122", Mac:"52:54:c6:21:db:29", Gw:"192.168.123.1", TapName:""} I0412 08:07:26.717495 16553 provision.go:415] Pod[hello121] adding resource to sandbox I0412 08:07:26.717577 16553 container.go:866] Pod[hello121] Con[bcd0082849f0(hello121)] begin add to sandbox I0412 08:07:26.717591 16553 volume.go:187] Pod[hello121] Vol[etchosts-volume] subcribe volume insert I0412 08:07:26.717597 16553 volume.go:200] Pod[hello121] Vol[etchosts-volume] subscribe the volume I0412 08:07:26.717607 16553 container.go:888] Pod[hello121] Con[bcd0082849f0(hello121)] finished container prepare, wait for volumes panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x6a8bca] goroutine 7608 [running]: panic(0x138f3a0, 0xc420012060) /usr/local/go/src/runtime/panic.go:500 +0x1a1 github.com/hyperhq/hyperd/vendor/github.com/hyperhq/runv/hypervisor.(*Vm).AddNic(0x0, 0xc421937f10, 0x0, 0x0)
On hykins, pod might take long time to start and hyperstart watcher times out. Then hyperd ignores the error and proceeds adding new containers, and eventually crashes.