Open jwinarske opened 1 year ago
In tmp/work-shared/raspberrypi4-64/kernel-source/.kernel-meta/cfg/merge_config_build.log
I'm seeing:
Merging .kernel-meta/configs//features/kvm/qemu-kvm.cfg
Value of CONFIG_KVM is redefined by fragment .kernel-meta/configs//features/kvm/qemu-kvm.cfg:
Previous value: CONFIG_KVM=y
New value: CONFIG_KVM=m
Value of CONFIG_TUN is redefined by fragment .kernel-meta/configs//features/kvm/qemu-kvm.cfg:
Previous value: CONFIG_TUN=m
New value: CONFIG_TUN=y
Value of CONFIG_MACVLAN is redefined by fragment .kernel-meta/configs//features/kvm/qemu-kvm.cfg:
Previous value: CONFIG_MACVLAN=y
New value: CONFIG_MACVLAN=m
Merging .kernel-meta/configs//cfg/fs/vfat.cfg
Merging .kernel-meta/configs//features/kvm/qemu-kvm.cfg
Merging .kernel-meta/configs//cfg/virtio.cfg
Value of CONFIG_VSOCKETS is redefined by fragment .kernel-meta/configs//cfg/virtio.cfg:
Previous value: CONFIG_VSOCKETS=m
New value: CONFIG_VSOCKETS=y
make[1]: Entering directory '/__w/meta-flutter/rpi4-64-vulkan-kirkstone/raspberrypi4-64/tmp/work/raspberrypi4_64-poky-linux/linux-raspberrypi/1
_5.15.34+gitAUTOINC+e1b976ee4f_0086da6acd-r0/linux-raspberrypi4_64-preempt-rt-build'
GEN Makefile
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/menu.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
./.tmp.config.56xptw7WlG:1619:warning: unexpected data:
./.tmp.config.56xptw7WlG:1631:warning: symbol value 'm' invalid for KVM
./.tmp.config.56xptw7WlG:1639:warning: unexpected data:
...
Value requested for CONFIG_KVM not in final .config
Requested value: CONFIG_KVM=m
Actual value: # CONFIG_KVM is not set
Value requested for CONFIG_KVM_INTEL not in final .config
Requested value: CONFIG_KVM_INTEL=m
Actual value:
Value requested for CONFIG_KVM_AMD not in final .config
Requested value: CONFIG_KVM_AMD=m
Actual value:
In tmp/work/raspberrypi4_64-poky-linux/linux-raspberrypi/1_5.15.34+gitAUTOINC+e1b976ee4f_0086da6acd-r0/linux-raspberrypi4_64-preempt-rt-build/.config
I see, which explains why kernel-module-kvm
is not valid:
CONFIG_VIRTUALIZATION=y
# CONFIG_KVM is not set
CONFIG_VHOST_IOTLB=m
CONFIG_VHOST=m
CONFIG_VHOST_MENU=y
CONFIG_VHOST_NET=m
CONFIG_VHOST_VSOCK=m
CONFIG_VHOST_CROSS_ENDIAN_LEGACY=y
CONFIG_TUN=y
CONFIG_TAP=m
virtual/kernel bcm2711_defconfig:
CONFIG_KVM=y
meta-virtualization features/kvm/qemu-kvm.cfg
augmentation of bcm2711_defconfig:
CONFIG_KVM=m
CONFIG_KVM_INTEL=m
CONFIG_KVM_AMD=m
Unclear what's causing this error, unless it can't be built as a module. Which could be resolved with another fragment.
Value requested for CONFIG_KVM not in final .config
Requested value: CONFIG_KVM=m
Actual value: # CONFIG_KVM is not set
I played around with patching a fragment, and I can't get it to patch after features/kvm/qemu-kvm-enable.scc
. Regardless of layer order, do_kernel_metadata() seems to always apply kernel cache fragments last.
For preempt-rt
the fragment diff looks like:
CONFIG_ARM64_ERRATUM_834220=y
CONFIG_IRQ_BYPASS_MANAGER=y
CONFIG_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQFD=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_MMIO=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_HAVE_KVM_VCPU_RUN_PID_CHANGE=y
CONFIG_KVM_XFER_TO_GUEST_WORK=y
# CONFIG_NVHE_EL2_DEBUG is not set
CONFIG_PREEMPT_NOTIFIERS=y
CONFIG_MMU_NOTIFIER=y
CONFIG_INTERVAL_TREE=y
The diff is different for non preempt-rt
.
When I run menuconfig
it's after all the other fragments are applied, so really the above diff fragment needs to be applied last.
Any suggestions on applying fragments in a per-determined order?
@zeddii
There's a couple of things at play, we haven't done much testing of meta-virt as a KVM host .. and by that, I mean that i don't know of any testing that has been done.
As such, the KVM enablement fragment isn't correct for arm as "config KVM" is a menuconfig, versus being a tristate in other architectures.
We could fix the fix that invalid symbol error, by making either the KERNEL_FEATURE being set be arch specific, or by making the features/kvm/qemu-kvm-enable.scc have a conditional within the fragment itself.
Both of those take a bit longer to bubble through the various repos.
To the kernel meta-data question, It isn't that the merging applies kernel-cache fragments last, it applies KERNEL_FEATURES last. We've had this on the various lists a few times, and it should be in the manual now, but something that is put into. KERNEL_FEATURE by a kernel recipe/bbappend is something considered to be a "contract" between the kernel recipe requirement and the runtime. So it isn't supposed to be something that is overridden on the SRC_URI.
That being said, you can override it. You can have a KERNEL_FEATURE defined in a layer. Which is just of course a reference to a fragment that is provided by the layer (and that fragment is in the right search path to be found).
I managed to find the link to where this made it into the docs: https://docs.yoctoproject.org/kernel-dev/common.html#adding-recipe-space-kernel-features
So you'd define your fragment above, and then refer to it as a KERNEL_FEATURE, so that i can be applied last.
And finally another idea is that you can (in the short term) do a :remove on the qemu-kvm-enable.scc value in KERNEL_FEATURES, and have something in the SRC_URI defining similar values to what that fragment currently does (except obviously with CONFIG_KVM=y in your case).
Hope that helps.
@zeddii Very helpful. The documentation is exactly what I was missing. Thank you!
https://github.com/agherzan/meta-raspberrypi/pull/1141
This PR builds kvm as part of kernel. In addition kernel-module-intel
, and kernel-module-amd
don't apply. So building kvm-image-minimal is not compatible with arm64 targets.
Using tip of kirkstone for meta-raspberrypi (this repo) and meta-virtualization (yocto) I am unable to build
kvm-image-minimal
.Describe the results you expected: kernel-module-kvm to be built.
Additional details (revisions used, host distro, etc.): CI job: https://github.com/meta-flutter/meta-flutter/blob/jw/rpi4_virt/.github/workflows/kirkstone-rpi4-64-vulkan.yml