search

arighi / virtme-ng

Quickly build and run kernels inside a virtualized snapshot of your live system
GNU General Public License v2.0
460 stars 51 forks source link

Unable to insert kernel modules while running kernel in a separate chroot #141

Open comeillfoo opened 4 months ago

comeillfoo commented 4 months ago

I've encountered a broken symbolic link for /lib/modules/$kver if the --root points to a rootfs directory that is not the parent of the directory where the kernel sources are located. E.g. kernel sources at directory linux and rootfs is sibling directory to sources:

.
|-- linux
`-- rootfs

So, when running vng --root ../rootfs being at linux I have modules at linux/.virtme_mods/lib/modules/0.0.0 and virtme_link_mods will be set to /../linux/.virtme_mods/lib/modules/0.0.0 because of os.path.relpath(kernel.moddir, args.root) (virtme/commands/run.py#L991).

Hence, virtme-init tries to symlink /lib/modules/$kver outside of the provided rootfs.

As a workaround I add a disk image (using --disk) with the missing directory, mount it to /tmp and manually relink /lib/modules/$kver to /tmp/my_disk/.virtme_mods/lib/modules/0.0.0. But are there other solutions besides this?

jimc commented 4 months ago

hey, it seems from your investigations, youre willing to hack some.

vng uses qemu to run the kernel in a vm, the "--kdir ./" arg to it makes module-loading path relative to $PWD

If this isnt the cause itself, there could be other path-relative issues.

comeillfoo commented 4 months ago

hey, it seems from your investigations, youre willing to hack some.

vng uses qemu to run the kernel in a vm, the "--kdir ./" arg to it makes module-loading path relative to $PWD

If this isnt the cause itself, there could be other path-relative issues.

hi, in fact, I'm willing to test linux kernel on arm64. That's why I have to use not a root folder as rootfs.

I haven't checked it but it seems that varying --kdir won't work for me.

Because for example if kdir is ./linux and root is ./rootfs (via vng --arch arm64 --root ./rootfs --run ./linux ...) then after virtme/commands/run.py#L484 kernel.moddir would be set to ./linux/.virtme_mods/lib/modules/0.0.0. And os.path.relpath between kernel.moddir and args.root would return the same path outside of rootfs - ../linux/.virtme_mods/lib/modules/0.0.0.