Oak Containers: Error creating the launcher client: vmw_vsock_virtio_transport

smherwig commented 2 days ago

I'm working of the main branch (commit c5190c07aa8da7436760f39bb5de8e27afb44844).

I'm attempting to run the rust_hello_world_trusted_bundle container using the following command:

sudo RUST_LOG=debug ./target/x86_64-unknown-linux-gnu/release/oak_containers_launcher \
    --system-image oak_containers/system_image/target/image.tar.xz \
    --container-bundle target/rust_hello_world_trusted_bundle.tar \
    --vmm-binary $(which qemu-system-x86_64) \
    --stage0-binary generated/stage0_bin \
    --kernel oak_containers/kernel/target/bzImage \
    --initrd target/stage1.cpio \
    --ramdrive-size 5000000 \
    --vm-type sev-es

The kernel boot is aborting with the following log lines:

...
brd: module loaded
tun: Universal TUN/TAP device driver, 1.6
virtio_net virtio0: virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
virtio_net virtio0: probe with driver virtio_net failed with error -22
amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled
Freeing initrd memory: 2736K
xt_time: kernel timezone is -0000
NET: Registered PF_INET6 protocol family
Segment Routing with IPv6
In-situ OAM (IOAM) with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered PF_PACKET protocol family
Bridge firewalling registered
NET: Registered PF_VSOCK protocol family
vmw_vsock_virtio_transport virtio1: virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
vmw_vsock_virtio_transport virtio1: probe with driver vmw_vsock_virtio_transport failed with error -22
IPI shorthand broadcast: enabled
AVX2 version of gcm_enc/dec engaged.
AES CTR mode by8 optimization enabled
sched_clock: Marking stable (684319486, 70770429)->(810324125, -55234210)
Loading compiled-in X.509 certificates
Freeing unused decrypted memory: 2036K
Freeing unused kernel image (initmem) memory: 2064K
Write protecting the kernel read-only data: 16384k
Freeing unused kernel image (rodata/data gap) memory: 620K
Run /init as init process
mke2fs 1.47.0 (5-Feb-2023)
Creating filesystem with 1250000 4k blocks and 312624 inodes
Filesystem UUID: f95e73cd-d068-4f4c-8e2f-1a0dab659d34
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done 

EXT4-fs (ram0): mounted filesystem f95e73cd-d068-4f4c-8e2f-1a0dab659d34 r/w without journal. Quota mode: disabled.
Error: error creating the launcher client

Caused by:
    0: transport error
    1: error trying to connect: No such device (os error 19)
    2: No such device (os error 19)
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100
Kernel Offset: 0x8c00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

I think the key log lines are:

vmw_vsock_virtio_transport virtio1: virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
vmw_vsock_virtio_transport virtio1: probe with driver vmw_vsock_virtio_transport failed with error -22

Our setup is:

server: Supermicro CloudDC A+ Server AS -2015CS-TNR
cpu: 1x EPYC 7354P 32C/64T CPU. This is an EPYC 4th Generation AMD Genoa chip, and supports SEV-SNP.
kernel: 6.11.0-rc3-snp-host-85ef1ac03941. We followed AMD's Using SEV with AMD EPYC Processors guide to setup the BIOS, SEV firmware, and host kernel. Specifically, we're running AMD's kernel as per the instructions on AMD's sep-latest branch

Additionally, I manually had to sudo modprobe vhost_vsock.

Thanks. I appreciated any help.

smherwig commented 2 days ago

To further clarify, we are using the nix install of qemu:

$ which qemu-system-x86_64
/nix/store/6x9r5ghwq2a6j4r2gmqida8d5wd0ya91-qemu-host-cpu-only-9.0.1/bin/qemu-system-x86_64

$ qemu-system-x86_64 --version
QEMU emulator version 9.0.1
Copyright (c) 2003-2024 Fabrice Bellard and the QEMU Project developers

conradgrobler commented 2 days ago

@dingelish if I remember correctly you ran into a similar problem with QEMU. Can you remember how you managed to resolve it?

dingelish commented 2 days ago

@dingelish if I remember correctly you ran into a similar problem with QEMU. Can you remember how you managed to resolve it?

sure. please apply this patch to your qemu

https://github.com/dingelish/qemu/commit/876e262744a6483a3bcb37a07767e6c75f9cf4ee

project-oak / oak

Oak Containers: Error creating the launcher client: vmw_vsock_virtio_transport #5044