search

hyperhq / runv

Hypervisor-based Runtime for OCI
Apache License 2.0
828 stars 129 forks source link

Failed to create Container with docker #357

Closed keyolk closed 7 years ago

keyolk commented 7 years ago

Hello All I'm trying to use runv with docker but it failed to launch simple busybox container

below is my env

$ runv -v
runv version 0.6.2
$ docker -v
Docker version 1.12.1, build 23cf638
$ uname -a
Linux keyolk-book 4.4.2-6ph #1 SMP Mon Jun 13 20:01:22 KST 2016 x86_64 GNU/Linux

what I did 

$ runv-containerd --debug --driver libvirt --kernel /var/lib/hyper/kernel --initrd /var/lib/hyper/hyper-initrd.img
$ docker daemon -D -l debug --containerd=/run/runv-containerd/containerd.sock
$ docker run -ti busybox /bin/sh

logs from runv-contaienrd

Libvirt Driver Loaded.
I1007 12:52:40.166856   12305 main.go:173] containerd: grpc api on /run/runv-containerd/containerd.sock
I1007 12:54:38.371208   12305 server.go:31] gRPC handle CreateContainer
I1007 12:54:38.376086   12305 hyperpod.go:337] Using kernel: ; Initrd: ;
I1007 12:54:38.380782   12305 vm.go:162] hyperHandlePodEvent pod pod-YrNtYqWBTr, vm vm-AOZFGRviwP
I1007 12:54:38.381316   12305 libvirt.go:541] domainXML: <domain type="kvm"><name>vm-AOZFGRviwP</name><memory unit="Mi
B">128</memory><vcpu placement="static" current="1">1</vcpu><os supported="yes"><type arch="x86_64" machine="pc-i440fx
-2.0">hvm</type><kernel>/var/lib/hyper/kernel</kernel><initrd>/var/lib/hyper/hyper-initrd.img</initrd><cmdline>console
=ttyS0 panic=1 no_timer_check</cmdline></os><features><acpi></acpi></features><cpu mode="host-passthrough"></cpu><on_p
oweroff>destroy</on_poweroff><on_reboot>destroy</on_reboot><on_crash>destroy</on_crash><devices><emulator>/usr/bin/qem
u-system-x86_64</emulator><controller type="pci" index="0" model="pci-root"></controller><controller type="virtio-seri
al" index="0"><address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x00"></address></controller><contr
oller type="scsi" index="0" model="virtio-scsi"><address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0
x00"></address></controller><controller type="usb" model="none"></controller><filesystem type="mount" accessmode="squa
sh"><driver type="path"></driver><source dir="/var/run/hyper/vm-AOZFGRviwP/share_dir"></source><target dir="share_dir"
></target><address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x00"></address></filesystem><channel t
ype="unix"><source mode="bind" path="/var/run/hyper/vm-AOZFGRviwP/hyper.sock"></source><target type="virtio" name="sh.
hyper.channel.0"></target></channel><channel type="unix"><source mode="bind" path="/var/run/hyper/vm-AOZFGRviwP/tty.so
ck"></source><target type="virtio" name="sh.hyper.channel.1"></target></channel><console type="unix"><source mode="bin
d" path="/var/run/hyper/vm-AOZFGRviwP/console.sock"></source><target type="serial" port="0"></target></console><membal
loon model="virtio"><address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x00"></address></memballoon>
</devices><seclabel type="none"></seclabel></domain>
E1007 12:54:38.450109   12305 libvirt.go:116] connection is alive
E1007 12:54:38.464339   12305 libvirt.go:545] Fail to launch domain [Code-1] [Domain-10] internal error: qemu unexpectedly closed the monitor: 2016-10-07T03:54:38.435765Z qemu-system-x86_64: -chardev socket,id=charserial0,path=/var/run/
hyper/vm-AOZFGRviwP/console.sock,server,nowait: Failed to bind socket to /var/run/hyper/vm-AOZFGRviwP/console.sock: Pe
rmission denied
I1007 12:54:38.479892   12305 hypervisor.go:29] vm vm-AOZFGRviwP: main event loop got message 21(COMMAND_RUN_POD)
I1007 12:54:38.484215   12305 vm_states.go:443] got spec, prepare devices
I1007 12:54:38.489786   12305 vm_states.go:67] initial vm spec: {
        "hostname": "e805da724fda",
        "containers": [],
        "shareDir": "share_dir"
    }
I1007 12:54:38.493506   12305 context.go:252] VM vm-AOZFGRviwP: state change from INIT to 'STARTING'
I1007 12:54:38.497201   12305 hypervisor.go:29] vm vm-AOZFGRviwP: main event loop got message 0(EVENT_VM_START_FAILED)
I1007 12:54:38.502463   12305 vm_states.go:479] VM did not start up properly, go to cleaning up
I1007 12:54:38.506159   12305 vm_states.go:36] VM has exit...
I1007 12:54:38.511823   12305 context.go:239] no more device to release/remove/umount, quit
I1007 12:54:38.512097   12305 vm.go:275] Get the response from VM, VM id is vm-AOZFGRviwP!
I1007 12:54:38.517960   12305 hyperpod.go:370] StartPod fail: QEMU response data is nil
I1007 12:54:38.522825   12305 supervisor.go:168] createHyperPod() returns
E1007 12:54:48.468613   12305 tty.go:139] Cannot connect to tty socket dial unix /var/run/hyper/vm-AOZFGRviwP/tty.sock: connect: no such file or directory
E1007 12:54:48.468613   12305 init_comm.go:126] Cannot connect to hyper socket dial unix /var/run/hyper/vm-AOZFGRviwP/hyper.sock: connect: no such file or directory
E1007 12:54:48.468627   12305 init_comm.go:49] failed to connected to /var/run/hyper/vm-AOZFGRviwP/console.sock dial unix /var/run/hyper/vm-AOZFGRviwP/console.sock: connect: no such file or directory

logs from docker daemon

DEBU[0071] Calling GET /v1.24/containers/json?all=1
DEBU[0081] Calling POST /v1.24/containers/create
DEBU[0081] form data: {"AttachStderr":true,"AttachStdin":true,"AttachStdout":true,"Cmd":["sh"],"Domainname":"","Entrypoint":null,"Env":[],"HostConfig":{"AutoRemove":false,"Binds":null,"BlkioDeviceReadBps":null,"BlkioDeviceReadIOps":null,"BlkioDeviceWriteBps":null,"BlkioDeviceWriteIOps":null,"BlkioWeight":0,"BlkioWeightDevice":null,"CapAdd":null,"CapDrop":null,"Cgroup":"","CgroupParent":"","ConsoleSize":[0,0],"ContainerIDFile":"","CpuCount":0,"CpuPercent":0,"CpuPeriod":0,"CpuQuota":0,"CpuShares":0,"CpusetCpus":"","CpusetMems":"","Devices":[],"DiskQuota":0,"Dns":[],"DnsOptions":[],"DnsSearch":[],"ExtraHosts":null,"GroupAdd":null,"IOMaximumBandwidth":0,"IOMaximumIOps":0,"IpcMode":"","Isolation":"","KernelMemory":0,"Links":null,"LogConfig":{"Config":{},"Type":""},"Memory":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":-1,"NetworkMode":"default","OomKillDisable":false,"OomScoreAdj":0,"PidMode":"","PidsLimit":0,"PortBindings":{},"Privileged":false,"PublishAllPorts":false,"ReadonlyRootfs":false,"RestartPolicy":{"MaximumRetryCount":0,"Name":"no"},"SecurityOpt":null,"ShmSize":0,"UTSMode":"","Ulimits":null,"UsernsMode":"","VolumeDriver":"","VolumesFrom":null},"Hostname":"","Image":"busybox","Labels":{},"NetworkingConfig":{"EndpointsConfig":{}},"OnBuild":null,"OpenStdin":true,"StdinOnce":true,"Tty":true,"User":"","Volumes":{},"WorkingDir":""}
DEBU[0081] container mounted via layerStore: /home/keyolk/mount/flash/docker/overlay/bc6f27b4d87581867b82e68addb970d11b046b076ee06f2115c95e7fc773e3b3/merged
DEBU[0081] Calling POST /v1.24/containers/e805da724fdaf75cbd16aee9d971521a44db5d94e93e494a9979125bdab15513/attach?stderr=1&stdin=1&stdout=1&stream=1
DEBU[0081] attach: stdin: begin
DEBU[0081] attach: stdout: begin
DEBU[0081] attach: stderr: begin
DEBU[0081] Calling POST /v1.24/containers/e805da724fdaf75cbd16aee9d971521a44db5d94e93e494a9979125bdab15513/start
DEBU[0081] container mounted via layerStore: /home/keyolk/mount/flash/docker/overlay/bc6f27b4d87581867b82e68addb970d11b046b076ee06f2115c95e7fc773e3b3/merged
DEBU[0081] Assigning addresses for endpoint jovial_ride's interface on network bridge
DEBU[0081] RequestAddress(LocalDefault/172.17.0.0/16, <nil>, map[])
DEBU[0082] Assigning addresses for endpoint jovial_ride's interface on network bridge
DEBU[0082] Programming external connectivity on endpoint jovial_ride (ab0b96a868ae9a5c50f70116d4f4da4b6e51d0a647032189815f6c4949a0d6b9)
ERRO[0082] Create container failed with error: StartPod fail
DEBU[0082] attach: stdout: end
DEBU[0082] attach: stdin: end
DEBU[0082] attach: stderr: end
DEBU[0082] Closing buffered stdin pipe
DEBU[0082] Revoking external connectivity on endpoint jovial_ride (ab0b96a868ae9a5c50f70116d4f4da4b6e51d0a647032189815f6c4949a0d6b9)
DEBU[0082] Releasing addresses for endpoint jovial_ride's interface on network bridge
DEBU[0082] ReleaseAddress(LocalDefault/172.17.0.0/16, 172.17.0.2)
ERRO[0082] Handler for POST /v1.24/containers/e805da724fdaf75cbd16aee9d971521a44db5d94e93e494a9979125bdab15513/start returned error: StartPod fail

Anything do I have to check more ?

gao-feng commented 7 years ago

Hi,

Could you show the content of /etc/libvirt/qemu.conf? please make sure qemu is running with root permission.

below is the content of /etc/libvirt/qemu.conf on my box.

user = "root"
group = "root"
clear_emulator_capabilities = 0

remember to restart libvirtd after change the config. :)

keyolk commented 7 years ago

@gao-feng it solved! Thanks to comment

there was no any of configuration before. only commented lines were there.