threefoldtech / cloud-container

A builder for a simple initramfs image to run container over virtiofs inside cloud-hypervisor
Apache License 2.0
0 stars 1 forks source link

Kernel and initramfs generated by cloud-container do not boot #7

Closed archit3kt closed 2 years ago

archit3kt commented 2 years ago

Hi there,

I'm actually testing grid 3.0 and I am trying to create my own images for learning purposes, but also to be able to create an up to date k3s image whenever I want, so I was directed here by Ahmed (and also here)

On my ubuntu 20.04 host I was able to launch a VM with alpine rootfs, 5.14 kernel from here and virtiofsd-rs with cloud-hypervisor, following steps described here

But when I try your process, despite the kernel and initramfs creation seems OK, I've got this error message when booting the VM :

Error booting VM: VmBoot(KernelLoad(Elf(InvalidElfMagicNumber))) [2021-11-23T10:26:00Z ERROR virtiofsd_rs] Waiting for daemon failed: HandleRequest(PartialMessage)

Could you please help me ?

muhamadazmy commented 2 years ago

Yes, we had a fix to this that wasn't pushed to main unfortunately, we don't use bzImage anymore because it's not supported by latest version of cloud hypervisor. Can u please pull, rebuild your image and try again

archit3kt commented 2 years ago

Interesting results : This works on baremetal Ubuntu 20.04, but there is a lot of these error messages on ttys0 :

cloud-hypervisor: 812.392555ms: WARN:vmm/src/vm.rs:436 -- Guest PIO read to unregistered address 0x87 cloud-hypervisor: 822.836755ms: WARN:vmm/src/vm.rs:461 -- Guest PIO write to unregistered address 0x4e

is this normal ?

Also I have strange mounts :

Filesystem Size Used Available Use% Mounted on dev 123.9M 0 123.9M 0% /dev run 129.6M 0 129.6M 0% /run tmpfs 129.6M 0 129.6M 0% /dev/shm /dev/root 725.4G 94.1G 594.4G 14% /new_root /dev/vda 20.0G 3.6M 19.5G 0% /new_root/opt/data

and this error message :

process virtiofs mounts ERROR: Root device mounted successfully, but /sbin/zinit does not exist. Bailing out, you are on your own. Good luck.

I suppose the /sbin/zinit is supposed to chroot in the /new_root ?

On a virtualbox VM, it kernel panic on boot (although I can boot the VM as I told on my first post) :

warning: use of deprecated parameter '--socket': Please use the '--socket-path' option instead. cloud-hypervisor: 2.153330199s: WARN:vmm/src/vm.rs:436 -- Guest PIO read to unregistered address 0x87 [ 1.186336] kernel tried to execute NX-protected page - exploit attempt? (uid: 0) [ 1.188752] BUG: unable to handle page fault for address: ffff888011e1e100 [ 1.188752] #PF: supervisor instruction fetch in kernel mode [ 1.188752] #PF: error_code(0x0011) - permissions violation [ 1.188752] PGD 3801067 P4D 3801067 PUD 3802067 PMD 8000000011e001e3 [ 1.188752] Thread overran stack, or stack corrupted [ 1.188752] Oops: 0011 [#1] PREEMPT SMP NOPTI [ 1.188752] CPU: 0 PID: 67 Comm: cryptomgr_test Not tainted 5.12.9 #1 [ 1.188752] Hardware name: Cloud Hypervisor cloud-hypervisor, BIOS 0 [ 1.188752] RIP: 0010:0xffff888011e1e100 [ 1.188752] Code: 00 00 00 00 00 00 00 00 00 00 3f fc 76 46 00 00 00 00 c0 e5 e1 11 80 88 ff ff ff ff ff ff ff ff ff 7f 00 00 00 00 00 00 00 00 e0 e1 11 80 88 ff ff 00 00 00 00 01 00 00 00 14 01 00 00 00 00 [ 1.188752] RSP: 0018:ffffc90000697d28 EFLAGS: 00010046 [ 1.188752] RAX: ffff888003df5dc0 RBX: 0000000000000000 RCX: 0000000000000000 [ 1.188752] RDX: 0000000000000000 RSI: ffff888003f7ddc0 RDI: 0000000000000000 [ 1.188752] RBP: ffffffff817e2b40 R08: 0000000000000000 R09: 0000000000000000 [ 1.188752] R10: 0000000000000326 R11: 0000000000000000 R12: 0000000000000000 [ 1.188752] R13: 0000000000000000 R14: 0000000000000000 R15: ffff88800518dcf0 [ 1.188752] FS: 0000000000000000(0000) GS:ffff888011e00000(0000) knlGS:0000000000000000 [ 1.188752] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1.188752] CR2: ffff888011e1e100 CR3: 0000000002c12000 CR4: 00000000000506f0 [ 1.188752] Call Trace: [ 1.188752] ? pm_suspend_timer_fn+0x70/0x70 [ 1.188752] ? perf_poll+0xb0/0xb0 [ 1.188752] CR2: ffff888011e1e100 [ 1.188752] ---[ end trace 5c4c13fcbe8773bd ]--- [ 1.188752] RIP: 0010:0xffff888011e1e100 [ 1.188752] Code: 00 00 00 00 00 00 00 00 00 00 3f fc 76 46 00 00 00 00 c0 e5 e1 11 80 88 ff ff ff ff ff ff ff ff ff 7f 00 00 00 00 00 00 00 00 e0 e1 11 80 88 ff ff 00 00 00 00 01 00 00 00 14 01 00 00 00 00 [ 1.188752] RSP: 0018:ffffc90000697d28 EFLAGS: 00010046 [ 1.188752] RAX: ffff888003df5dc0 RBX: 0000000000000000 RCX: 0000000000000000 [ 1.188752] RDX: 0000000000000000 RSI: ffff888003f7ddc0 RDI: 0000000000000000 [ 1.188752] RBP: ffffffff817e2b40 R08: 0000000000000000 R09: 0000000000000000 [ 1.188752] R10: 0000000000000326 R11: 0000000000000000 R12: 0000000000000000 [ 1.188752] R13: 0000000000000000 R14: 0000000000000000 R15: ffff88800518dcf0 [ 1.188752] FS: 0000000000000000(0000) GS:ffff888011e00000(0000) knlGS:0000000000000000 [ 1.188752] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1.188752] CR2: ffff888011e1e100 CR3: 0000000002c12000 CR4: 00000000000506f0 [ 1.188752] note: cryptomgr_test[67] exited with preempt_count 2