search

Mirantis / virtlet

Kubernetes CRI implementation for running VM workloads
Apache License 2.0
743 stars 128 forks source link

Virtual machine instance file lost after kubernetes host reboot #786

Closed jiadexin closed 6 years ago

jiadexin commented 6 years ago

I deployed virtlet on a single-node kubenertes cluster. After creating a vm in k8s,it works fine , and the instance file is in the directory /var/lib/virtlet:

ll /var/lib/virtlet/volumes/ -hs 4.0K drwxr-xr-x 2 root root 4.0K October 17 16:08 ./ 4.0K drwxr-xr-x 5 root root 4.0K October 17 16:13 ../   20M -rw------- 1 64055 syslog 20M October 17 16:17 virtlet_root_7bfdd7f7-347b-5ed1-60b3-81cfc72ab1ba

but when i reboot the k8s node ,the vm files in the /var/lib/virtlet/volumes/ directory are missing:

ll /var/lib/virtlet/volumes/ -hs 4.0K drwxr-xr-x 2 root root 4.0K 10月 17 16:19 ./ 4.0K drwxr-xr-x 5 root root 4.0K 10月 17 16:18 ../

and the vm pod start failed:

kubectl describe po master-vm-0 .... Events: Type Reason Age From Message Normal Scheduled 12m default-scheduler Successfully assigned default/master-vm-0 to jiadx-m4800 Normal Started 8m (x2 over 12m) kubelet, jiadx-m4800 Started container Warning Failed 2m kubelet, jiadx-m4800 Error: "/run/virtlet.sock": rpc error: code = 2 desc = failed to look up domain "12460e7b-34b6-5828-432a-f8154967d5f2": domain not found Warning Failed 2m kubelet, jiadx-m4800 Error: "/run/virtlet.sock": rpc error: code = 2 desc = failed to create domain "12460e7b-34b6-5828-432a-f8154967d5f2": virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: I1017 08:18:53.883789 23244 vmwrapper.go:66] Obtaining PID of the VM container process... E1017 08:18:53.884049 23244 vmwrapper.go:89] Failed to obtain tap fds for key "d8cecb56-d1e3-11e8-95dd-f01faf5ead4d": bad oob data size: 0 instead of 16') Warning Failed 2m kubelet, jiadx-m4800 Error: "/run/virtlet.sock": rpc error: code = 2 desc = failed to create domain "12460e7b-34b6-5828-432a-f8154967d5f2": virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: I1017 08:18:56.132471 23366 vmwrapper.go:66] Obtaining PID of the VM container process... E1017 08:18:56.132804 23366 vmwrapper.go:89] Failed to obtain tap fds for key "d8cecb56-d1e3-11e8-95dd-f01faf5ead4d": bad oob data size: 0 instead of 16') Warning Failed 2m kubelet, jiadx-m4800 Error: "/run/virtlet.sock": rpc error: code = 2 desc = failed to create domain "12460e7b-34b6-5828-432a-f8154967d5f2": virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: I1017 08:18:58.385572 23467 vmwrapper.go:66] Obtaining PID of the VM container process... `

How should I solve it?

jiadexin commented 6 years ago

Same as the https://github.com/Mirantis/virtlet/issues/783