Guest fails to mount root FS

[zevweiss]

This may well just be me missing something obvious, but I'm running into this on attempting to boot a guest with virtme-run --installed-kernel --pwd:

[    0.339347] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -19
[    0.340175] Please append a correct "root=" boot option; here are the available partitions:
[    0.341045] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

Using --show-command yields:

/usr/local/bin/qemu-system-x86_64 -fsdev local,id=virtfs1,path=/,security_model=none,readonly -device virtio-9p-pci,fsdev=virtfs1,mount_tag=/dev/root -fsdev local,id=virtfs5,path=/lib/modules/4.1.1+,security_model=none,readonly -device virtio-9p-pci,fsdev=virtfs5,mount_tag=virtme.moddir -machine accel=kvm:tcg -watchdog i6300esb -cpu host -parallel none -net none -echr 1 -serial none -chardev stdio,id=console,signal=off,mux=on -serial chardev:console -mon chardev=console -vga none -display none -kernel /boot/vmlinuz-4.1.1+ -append 'init=/home/zev/src/virtme/virtme/guest/virtme-init earlyprintk=serial,ttyS0,115200 console=ttyS0 psmouse.proto=exps "virtme_stty_con=rows 77 cols 119 iutf8" TERM=screen virtme_chdir=home/zev/src/virtme rootfstype=9p rootflags=ro,version=9p2000.L,trans=virtio,access=any raid=noautodetect'

This is on CentOS 6 with Python 3.5.0, qemu 2.4.0.1, and a 4.1-based kernel. Any tips would be appreciated.

[amluto]

Hmm, sounds like it's screwing up the initramfs decision.

Can you check if /lib/modules/uname -r exists? Does adding --force-initramfs work better?

On Fri, Oct 16, 2015 at 10:56 AM, Zev Weiss notifications@github.com wrote:

This may well just be me missing something obvious, but I'm running into this on attempting to boot a guest with virtme-run --installed-kernel --pwd:

[ 0.339347] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -19 [ 0.340175] Please append a correct "root=" boot option; here are the available partitions: [ 0.341045] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

Using --show-command yields:

/usr/local/bin/qemu-system-x86_64 -fsdev local,id=virtfs1,path=/,security_model=none,readonly -device virtio-9p-pci,fsdev=virtfs1,mount_tag=/dev/root -fsdev local,id=virtfs5,path=/lib/modules/4.1.1+,security_model=none,readonly -device virtio-9p-pci,fsdev=virtfs5,mount_tag=virtme.moddir -machine accel=kvm:tcg -watchdog i6300esb -cpu host -parallel none -net none -echr 1 -serial none -chardev stdio,id=console,signal=off,mux=on -serial chardev:console -mon chardev=console -vga none -display none -kernel /boot/vmlinuz-4.1.1+ -append 'init=/home/zev/src/virtme/virtme/guest/virtme-init earlyprintk=serial,ttyS0,115200 console=ttyS0 psmouse.proto=exps "virtme_stty_con=rows 77 cols 119 iutf8" TERM=screen virtme_chdir=home/zev/src/virtme rootfstype=9p rootflags=ro,version=9p2000.L,trans=virtio,access=any raid=noautodetect'

This is on CentOS 6 with Python 3.5.0, qemu 2.4.0.1, and a 4.1-based kernel. Any tips would be appreciated.

— Reply to this email directly or view it on GitHub https://github.com/amluto/virtme/issues/5.

[zevweiss]

/lib/modules/uname -r does exist, yes.

With --force-initramfs I get:

[    0.343275] virtme initramfs: mounting hostfs...
virtme: initramfs does not have module fs-9p
mount: mounting /dev/root on /newroot/ failed: No such device
Failed to switch to real root.  We are stuck.

I have (what I think are) all the relevant modules mentioned in the README loaded:

[zev@adsl-z01: virtme]% lsmod | egrep '9p|virtio'
virtio_scsi            11168  0 
virtio_net             23336  0 
9pnet_virtio            7646  0 
9p                     47552  0 
9pnet                  70175  2 9pnet_virtio,9p
fscache                55935  1 9p
virtio_pci             15214  0 
virtio_ring             9409  4 virtio_scsi,virtio_net,9pnet_virtio,virtio_pci
virtio                  7007  4 virtio_scsi,virtio_net,9pnet_virtio,virtio_pci

(Though I guess those are probably more relevant for the guest I suppose...)

[zevweiss]

Hmm, some sort of module-alias confusion?

[zev@adsl-z01: virtme]% fgrep fs-9p /lib/modules/`uname -r`/modules.alias
alias fs-9p 9p

[amluto]

Internally, virtme (host-side) is doing:

$ modprobe --show-depends -C /var/empty -S `uname -r` fs-9p 2>/dev/null
insmod /lib/modules/4.1.10-200.fc22.x86_64/kernel/fs/fscache/fscache.ko.xz 
insmod /lib/modules/4.1.10-200.fc22.x86_64/kernel/net/9p/9pnet.ko.xz 
insmod /lib/modules/4.1.10-200.fc22.x86_64/kernel/fs/9p/9p.ko.xz

On my system, that works. Can you see what the same command outputs for you?

virtme doesn't bother with aliases on the guest side -- it just precomputes the entire list of modules on the host and loads them all on the guest.

[zevweiss]

Ah, that might explain it...my modprobe (module-init-tools version 3.9) doesn't support the -S flag, so that just fails with nothing sent to stdout.

[zevweiss]

Hmm -- and I see my man page claims it supports -S, but trying to actually use it just fails with an "invalid option" error...sigh.

[amluto]

That's unfortunate. Try d4777bd5eae0 -- it might help as long as the kernel you're booting is the running kernel.

[zevweiss]

Excellent -- that seems to do the trick. It now appears to be pretty much working properly, though I do see some unexpected (to me, at least) error messages from mount and cp as the guest boots:

[    0.427457] virtme initramfs: mounting hostfs...
mount: mounting run on /newroot/run failed: No such file or directory
cp: cannot create directory '/newroot/run/virtme': No such file or directory
[    0.432466] virtme initramfs: done; switching to real root
[    0.778924] virtme-init: basic initialization done

This happens both with and without --force-initramfs -- not sure if it's anything to be concerned about; I do end up with a working console & shell despite this. Feel free to close this issue if you want, though I'll leave it open at least for now in case that's something you want to look into further.

[amluto]

Can you send the output of 'ls -l /' and 'mount' in the guest? Something's probably a bit broken.

On Fri, Oct 16, 2015 at 1:25 PM, Zev Weiss notifications@github.com wrote:

Excellent -- that seems to do the trick. It now appears to be pretty much working properly, though I do see some unexpected (to me, at least) error messages from mount and cp as the guest boots:

[ 0.427457] virtme initramfs: mounting hostfs... mount: mounting run on /newroot/run failed: No such file or directory cp: cannot create directory '/newroot/run/virtme': No such file or directory [ 0.432466] virtme initramfs: done; switching to real root [ 0.778924] virtme-init: basic initialization done

This happens both with and without --force-initramfs -- not sure if it's anything to be concerned about; I do end up with a working console & shell despite this. Feel free to close this issue if you want, though I'll leave it open at least for now in case that's something you want to look into further.

— Reply to this email directly or view it on GitHub https://github.com/amluto/virtme/issues/5#issuecomment-148824750.

[amluto]

Oh, duh. You probably just have an old enough distro that /run doesn't exist. Try 28bafbf8e954.

[zevweiss]

OK, great -- no more error messages now, everything looks clean.

(Though as a slight addendum to that commit message, I should note that the distro-provided qemu from CentOS 6 didn't work, since it doesn't provide qemu-system-* binaries in $PATH -- I installed a newer one from source, and likewise with Python 3.)

Anyway, thanks for the prompt & helpful response -- looking forward to using this for (hopefully) smoother/easier kernel debugging.