agherzan / meta-raspberrypi

Yocto/OE BSP layer for the Raspberry Pi boards
https://www.yoctoproject.org/
MIT License
534 stars 410 forks source link

Enable KVM for raspberrypi4-64 #1141

Closed jwinarske closed 1 year ago

jwinarske commented 1 year ago

-removes kernel-cache qemu-kvm-enable.scc -locally set KVM kernel config

Signed-off-by: Joel Winarske joel.winarske@gmail.com

agherzan commented 1 year ago

Could you add some context on this issue and why this ie required on the BSP layer?

jwinarske commented 1 year ago

@agherzan

Technical details here: https://github.com/agherzan/meta-raspberrypi/issues/1139

Effectively meta-virtualization is overriding the default kernel config, which ends up with a kernel config that prevents kvm from being built.

Given it's a platform specific change, this layer is the correct place for it.

jwinarske commented 1 year ago

@kraj

thanks for making meta-raspberrypi better. I think it would be good to look into kernel-config-cache to see what we can reuse here. E.g. I do see features/kvm/qemu-kvm-enable.scc [1] [2] in kernel-meta repo. So can we try using that ?

features/kvm/qemu-kvm-enable.scc gets added when kvm is added to DISTRO_FEATURES. This is the root cause of kvm not being built, as features/kvm/qemu-kvm-enable.scc is not a compatible kernel config for linux-raspberrypi, as such causes kvm to be disabled.

This change is frag diff post meta-virtualization + DISTRO_FEATURES:append = ' kvm virtualization'.

Notice I am removing features/kvm/qemu-kvm-enable.scc here

Please see #1139 for further info.

kraj commented 1 year ago

@kraj

thanks for making meta-raspberrypi better. I think it would be good to look into kernel-config-cache to see what we can reuse here. E.g. I do see features/kvm/qemu-kvm-enable.scc [1] [2] in kernel-meta repo. So can we try using that ?

features/kvm/qemu-kvm-enable.scc gets added when kvm is added to DISTRO_FEATURES. This is the root cause of kvm not being built, as features/kvm/qemu-kvm-enable.scc is not a compatible kernel config for linux-raspberrypi, as such causes kvm to be disabled.

This change is frag diff post meta-virtualization + DISTRO_FEATURES:append = ' kvm virtualization'.

Notice I am removing features/kvm/qemu-kvm-enable.scc here

Please see #1139 for further info.

hmm. intertesting, can we see if we can improve yocto-kernel-cache here or is that not a good idea?

agherzan commented 1 year ago

Right. Have we found out what makes the cache config incompatible with the one provided in the RPI fork? Would we be able to inject a cfg that fixes the incompatibility before applying the cache KVM cfg? Or even try to improve the cache cfg to cater for that?

jwinarske commented 1 year ago

@kraj I'm suspecting putting linux-raspberrypi specific flags in yocto-kernel-cache would be frowned upon. I can try the same flags on another aarch64 target, but I'm thinking there may be some issues. Factoring out the lowest common denominator in these things is extremely time consuming, as I suspect you know :)

@agherzan Yes. The incompatibility directly comes from the meta-virtualization/kernel-config-cache fragment features/kvm/qemu-kvm-enable.scc. This PR replaces features/kvm/qemu-kvm-enable.scc with correct combo. One could sort out patching features/kvm/qemu-kvm-enable.scc, but I figured that was too messy of a solution. I think it's clearer to see the platform specific change locally.

I'm working on getting crosvm running for AGL. Recipes here.

jwinarske commented 1 year ago

ping

agherzan commented 1 year ago

Thanks, @jwinarske for the details. I feel uneasy to get this in for two reasons. We propose to push in meta-raspberrypi a bunch of configs that we just know work for the KVM use case but we don't know what configs are the problem when using the upstream fragments. Also, we pretty much hide the issue upstream as opposed to fixing it there or bringing in the BSP layer only for the dry fix/adaptation. On top of that, I have to make sure this is maintained in the future which I'm happy to do as long as we narrow it down to the RPi-specific part of it. So I hope that we can debug this further and get to the bottom of this config dependency. Once we do, we can distil a config that caters for the RPi kernel deviation (if at all).

jwinarske commented 1 year ago

@agherzan it's very clear what the conflicting fragments are. Upstream enables three flags, two of which are Intel and AMD, the other build kvm as module. All conflict. We can selectively replace feature fragments based on Bruce's input which I am doing. Given linux-raspberrypi is a kernel fork, locating this fragment in meta-virtualization would only make sense if another aarch64 platform requires the exact same flags.

I will look into a couple other aarch64 platforms. If they have identical flags, then I will pursue a patch to meta-virtualization.

I will report back.

agherzan commented 1 year ago

Thanks, @jwinarske. Looking forward to your findings.

jwinarske commented 1 year ago

@agherzan my bad I updated this branch without realizing it was part of the PR. I think we can close this for now. If I have more time I will pursue.

agherzan commented 1 year ago

No worries. Feel free to come back when you get a chance.