search

ublue-os / bluefin

The next generation Linux workstation, designed for reliability, performance, and sustainability.
https://projectbluefin.io
Apache License 2.0
1.26k stars 163 forks source link

Bootc-native way to do configure-vfio or remove it? #1966

Open tulilirockz opened 3 days ago

tulilirockz commented 3 days ago

We need to find some more integrated way to do the configure-vfio thing, either by having a separate image for it, including it on the main images, or just getting rid of it entirely as we are trying to move to a bootc-only future.

tulilirockz commented 3 days ago

configure-vfio enables amd_iommu, sets iommu to passthrough mode, and makes it so you can add your PCI-E devices to the kargs after that with rpm-ostree kargs --append-if-missing. This is literally just straight up not supported on bootc (and we cant ship Bluefin w/ amd_iommu=on / iommu=pt by default.)

tulilirockz commented 3 days ago

I believe sysexts also cant really help out with this either

tulilirockz commented 3 days ago

The fedora docs mention that "When using bootc install, it is possible to add install-time kernel arguments both in the container image, and via bootc install --karg. "Day 2 updatable" kernel arguments are supported as well."

tulilirockz commented 3 days ago

Oh. We could change it into a service that automatically does this weird workaround mentioned on https://docs.fedoraproject.org/en-US/bootc/kernel-args/. - Although to be completely honest I feel like this shouldn't be supported by Bluefin directly, we should just let it be a separate project or something that sets up that.

HikariKnight commented 3 days ago

Kargs are the easiest i would say since we need that in bazzite anyway but the main issue is enabling the vfio modules as they are disabled by default and has to be enabled and added to the initramfs

vfio
vfio_iommu_type1
vfio-pci

for bluefin and aurora obviously this can be done and enabled on the dx images if feasable as that makes sense. I have not found any info as to why the vfio modules are disabled by default though.

the kvmfr static_size_mb modprobe config can easily be moved to a karg with kvmfr.static_size_mb=128 as that module is active anyway with 0mb loaded into memory by default. its just a modprobe file today because its in-line with the looking-glass documentation so it is easier for the user to edit it, but moving it to a karg is a non-issue.