Failure to run kata containers+ firecracker with kubernetes.

[I-m2310]

Kata containers and Qemu run on the same cluster without problem, but not firecracker.

Versions --->
Ubuntu server 18.04.3 LTS
kernel 4.15.0-60-generic
kubernetes: 1.15.3
firecracker 0.18.0
cri-containerd : 1.2.9
kata-runtime  : 1.9.0-alpha0
   commit   : <<unknown>>
   OCI specs: 1.0.1-dev

The config file ->

cat /etc/kata-containers/configuration.toml
[hypervisor.firecracker]
path = "/usr/bin/firecracker"
kernel = "/usr/share/kata-containers/vmlinuz.container"
image = "/usr/share/kata-containers/kata-containers.img"
block_device_driver = "virtio-mmio"
enable_debug = true
use_vsock = true
kernel_params = ""
firmware = ""
default_vcpus = 1
default_maxvcpus = 0
default_bridges = 1
default_memory = 4096
disable_block_device_use = false

[factory]

[proxy.kata]
path = "/usr/libexec/kata-containers/kata-proxy"

[shim.kata]
path = "/usr/libexec/kata-containers/kata-shim"

[agent.kata]

[netmon]
path = "/usr/libexec/kata-containers/kata-netmon"

[runtime]
internetworking_model="tcfilter"
disable_guest_seccomp=true

The firecracker pod has been stuck at the "ContainerCreating" status. It returns the following error:

Failed create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: [PUT /vsocks/{id}][400] putGuestVsockByIdBadRequest &{FaultMessage:unknown fieldid, expected one ofvsock_id,guest_cid,uds_pathat line 1 column 28}: unknown

[andreeaflorescu]

Are you using the latest Firecracker version? We changed the vsock implementation and there are some changes required on Kata side as well.

[I-m2310]

Thank you for the quick response.

That's right, I was using the latest version, 0.18.0.

I downgraded to firecracker 0.17.0 and I overcame this error but now I got a new one.

The new error -> Failed create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: [PUT /vsocks/{id}][400] putGuestVsockByIdBadRequest &{FaultMessage:Invalid request method and/or path: PUT /vsocks/root}: unknown

Kernel modules ->

alias char_major_10_238 vhost_net
alias char_major_10_241 vhost_vsock
alias devname:vhost_net vhost_net
alias devname:vhost_vsock vhost_vsock
alias tcm_vhost vhost_scsi
alias symbol:vhost_add_used vhost
alias symbol:vhost_add_used_and_signal vhost
alias symbol:vhost_add_used_and_signal_n vhost
alias symbol:vhost_add_used_n vhost
.....

And

  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
..... 

[dhrgit]

@I-m2310 Judging by the error you are getting ("Invalid request and/or path"), it looks like the Firecracker binary you are using was built without experimental vsock support. In Firecracker versions prior to 0.18.0, vsock was included as an experimental, compile-time feature that was not enabled by default. In order to use vsock, Firecracker needed to be built with --feature vsock (i.e. devtool build --release -- --feature vsock).

[I-m2310]

Thank you ! Building firecracker with --feature vsock, fixed the error.