Open jgcodes2020 opened 2 years ago
Most of you probably already know this but before getting too excited about mainline SR-IOV I would read this post by an AMD developer.
SR-IOV virtual functions (VFs) are not the same as physical functions (PFs). You need explicit support for the VF in the KMDs for every OS you want to run in a VM. You can't just use the regular bare metal drivers.
He is basically suggesting that AMD's Windows drivers don't support SR-IOV, thus making the whole thing pointless for most of us. If the same stands true for the Intel Windows drivers you might not have an use case case for a mainline SR-IOV kernel.
Most of you probably already know this but before getting too excited about mainline SR-IOV I would read this post by an AMD developer.
SR-IOV virtual functions (VFs) are not the same as physical functions (PFs). You need explicit support for the VF in the KMDs for every OS you want to run in a VM. You can't just use the regular bare metal drivers.
He is basically suggesting that AMD's Windows drivers don't support SR-IOV, thus making the whole thing pointless for most of us. If the same stands true for the Intel Windows drivers you might not have an use case case for a mainline SR-IOV kernel.
Who cares about using this with windows
Who cares about using this with windows
I would love to use it to virtualize OSX when I need to target such platform but apparently this is not an option.
@darkbasic https://github.com/intel/linux-intel-lts/issues/33#issuecomment-1326178155 <- right here in this thread someone got it to work fine on windows so I'd assume that Intel's drivers support it?
@darkbasic #33 (comment) <- right here in this thread someone got it to work fine on windows so I'd assume that Intel's drivers support it?
Not a chance. macOS might soon drop Intel support altogether and there's not a single Mac with Intel Xe graphics. Unless the community makes the drivers somehow, we'll never have it. Even nVidia GPUs aren't supported since High Sierra.
But the Windows side works fine. With this specific kernel.
@darkbasic #33 (comment) <- right here in this thread someone got it to work fine on windows so I'd assume that Intel's drivers support it?
Not a chance. macOS might soon drop Intel support altogether and there's not a single Mac with Intel Xe graphics. Unless the community makes the drivers somehow, we'll never have it. Even nVidia GPUs aren't supported since High Sierra.
But the Windows side works fine. With this specific kernel.
i mean, yeah windows drivers DO support it, and that's what most people care about I assume. Shucks about macOS, but it'd be dumb to expect macOS to support this properly
ping, any progress?
yeah, wasnt this supposed to land in q4 2023? This is a pretty nice feature that friends I have would love to use, but... not yet mainstreamed.
ya what happened?
i mean, there was a comment above saying it's postponed due to drm interface changes. However no one said for how long. and until this year I hoped it will not be that long :)
I read somewhere that the kernel 6.8 will have this, but seems like that's not happening?
If this gets merged (or if it already is?) there seems to be something regarding VF ... https://gitlab.freedesktop.org/drm/xe/kernel/-/commits/drm-xe-next
Tangentially related: anyone know if there are there any interim software based solutions for partitioning 13th/14th gen Intel iGPUs until this lands? I just want to be able to pass the hardware encoding capabilities to more than one VM 😅
Tangentially related: anyone know if there are there any interim software based solutions for partitioning 13th/14th gen Intel iGPUs until this lands? I just want to be able to pass the hardware encoding capabilities to more than one VM 😅
Kernel 6.1 or whatever older that is known/confirmed working in this thread or similar...
I am testing the Intel Xe driver from kernel 6.8 RC5, even got mesa-git and all that, and it can't even show a desktop without drawing GPU rainbow art all over it. But I bet I am doing something wrong as Phoronix already ran tests on it. But nothing was announced, it is experimental, so I had no expectations.
Still, until the Xe driver matures, they should've at least got i915 working reliably (ie no suspend issues) in mainline. I wish I could sell my laptop at this point and buy something older that at least has gvt-g working. They removed a feature for which they don't have a replacement 2 years later. Basically these GPUs have sriov support, but it was never made properly functional in any software. Hyper-V has it partially working, but it's apparent it was more focused on Windows Sandbox and WSL-g.
Any updates on SR-IOV and kernel 6.8? It's getting rolled into Proxmox 8.2, and I'd really like to be able to use SR-IOV with my Alder Lake i5-12500T iGPU without having to do the dkms-based wizardry here (that has to be done every time Proxmox updates the kernel at all...which is a lot).
If the same stands true for the Intel Windows drivers you might not have an use case case for a mainline SR-IOV kernel.
@darkbasic SR-IOV seems to work fine with Intel's Windows drivers. I have an Unraid server on Linux 6.1.x, running a Windows Server 2022 VM for Blue Iris, and have no trouble using hardware accelerated video encoding/decoding in the VM. I haven't tried anything graphically intensive, but I can definitely see the iGPU being used for video encoding/decoding.
Thanks. :)
I see these updates, and they're always "preparing for SR-IOV" and talking about how once released the Xe driver won't be the default. Given that these news releases focus on Xe2-based GPUs and other newer hardware, I'm not quite sure how to predict what we'll actually get.
If Xe is not the default, but complete, should we just be able to switch to it somehow (instead of using i915) and have SR-IOV just start working so long as its enabled in the BIOS and everything else is configured correctly?
I'm trying to get a better understanding of when the current DKMS approach won't be needed anymore for 12th+ gen iGPUs.
It's been over 2 years since I first opened this issue, can someone from Intel give us an update as to which drivers (Xe or i915) will support SR-IOV and when?
I really wish they'd surprise us and have this enabled on the A770 but I'm not counting on it.
It's been over 2 years since I first opened this issue, can someone from Intel give us an update as to which drivers (Xe or i915) will support SR-IOV and when?
You might have seen it already, but the DKMS module for i915 SR-IOV support was recently updated to support kernel versions up to 6.9: https://github.com/strongtz/i915-sriov-dkms. It previously only supported up to 6.5.
It's been over 2 years since I first opened this issue, can someone from Intel give us an update as to which drivers (Xe or i915) will support SR-IOV and when?
You might have seen it already, but the DKMS module for i915 SR-IOV support was recently updated to support kernel versions up to 6.9: https://github.com/strongtz/i915-sriov-dkms. It previously only supported up to 6.5.
Yes. I'm hoping. to test this soon. :)
Meanwhile: https://www.phoronix.com/news/Intel-Xe-DRM-Next-Linux-6.12 Intel is targeting Kernel 6.12 for "Intel Xe2 Lunar Lake & Battlemage Graphics," and also "further support for SR-IOV."
Hello!
The latest kernel tree from intel seems to work with SR-IOV on A770. It is based on the 6.11.0-rc2 kernel version, you might want to compile this manually.
You can find this kernel tree here:
https://gitlab.freedesktop.org/drm/xe/kernel
This is what i used for my configs:
/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on kvm.ignore_msrs=1 vfio_pci.ids=8086:56a0,8086:4f90 pcie_aspm=off"
/etc/modprobe.d/blacklist.conf
blacklist i915
blacklist xe
blacklist snd_hda_intel
@jonas5 nice, but you blacklist xe? Did you compile xe as built-in instead of a module?
This is great news!
Hopefully this will eventually come to the 12th and 13th gen iGPUs, as well. I would love to be able to use SR-IOV on Proxmox without a DKMS kernel driver.
From this and some other things I’ve read, it seems like they’re shooting for 6.12 to have this stable and working, if not default.
On Aug 27, 2024, at 10:46 AM, jonas5 @.***> wrote:
Hello!
The latest kernel tree from intel seems to work with SR-IOV on A770. It is based on the 6.11.0-rc2 kernel version, you might want to compile this manually.
You can find this kernel tree here: https://gitlab.freedesktop.org/drm/xe/kernel
Screenshot.from.2024-08-27.16-40-11.png (view on web) https://github.com/user-attachments/assets/79a6d85a-0692-400a-adbd-5d68ca011c4d This is what i used for my configs:
/etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on kvm.ignore_msrs=1 vfio_pci.ids=8086:56a0,8086:4f90 pcie_aspm=off"
/etc/modprobe.d/blacklist.conf
blacklist i915 blacklist xe blacklist snd_hda_intel — Reply to this email directly, view it on GitHub https://github.com/intel/linux-intel-lts/issues/33#issuecomment-2312924349, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGI5CYQ7XQ6XQLZHFDI6WK3ZTSNPFAVCNFSM5Z63XG72U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMZRGI4TENBTGQ4Q. You are receiving this because you commented.
The latest kernel tree from intel seems to work with SR-IOV on A770
That would be huge because Intel stated multiple times that A770 doesn't support SR-IOV. Can someone else please confirm this?
The latest kernel tree from intel seems to work with SR-IOV on A770
That would be huge because Intel stated multiple times that A770 doesn't support SR-IOV. Can someone else please confirm this?
https://www.phoronix.com/news/Intel-Xe-Driver-SR-IOV-Plans
https://lore.kernel.org/dri-devel/20231110182231.1730-1-michal.wajdeczko@intel.com/
SR-IOV Platforms
================
Initially we plan to add SR-IOV functionality to the following SDV platforms
already supported by the Xe driver:
- TGL (up to 7 VFs)
- ADL (up to 7 VFs)
- MTL (up to 7 VFs)
- ATSM (up to 31 VFs)
- PVC (up to 63 VFs)
Newer platforms will be supported later, but we hope that enabling will be
much faster, as majority of the driver changes are either platform agnostic
or are similar between earlier platforms (hence we start with SDVs).
Looks like Tigerlake, Alderlake, Meteorlake, Arctic Sound M, Ponte Vecchio respectively
It'd be nice to have some sort of official statement about where everything's at with this.
I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible
It'd be nice to have some sort of official statement about where everything's at with this.
I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible
I'd love to hear that too. Has anyone made it work? If so, how?
It'd be nice to have some sort of official statement about where everything's at with this.
I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible
As far as I know, they're targeting Linux 6.12. Someone earlier in the comments got it working with an A770 using 6.11.0-rc2.
Having said that, Raptor Lake is not in the initial support list that @adapt-L posted above.
It'd be nice to have some sort of official statement about where everything's at with this. I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible
As far as I know, they're targeting Linux 6.12. Someone earlier in the comments got it working with an A770 using 6.11.0-rc2.
Having said that, Raptor Lake is not in the initial support list that @adapt-L posted above.
Sorry, but doesn't these two statements conflict with each other? From what I've read, Intel Arc Alchemist dGPUs aren't on the list that @adapt-L posted either, or am I misreading that? And I'd assume that Raptor-Lake is closer to ADL (which is on the list) then Arc Alchemist dGPUs?
Other than that, I hope SR-IOV is finally mainlined with 6.12!
It'd be nice to have some sort of official statement about where everything's at with this. I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible
I'd love to hear that too. Has anyone made it work? If so, how?
I would like to know the answer to this question myself. I don't care 'if' it is supported but 'when' it is supported, how is this intended to be enabled? I am running the xe
driver, I have enabled iommu
I have set max_vfs
on the kernel command line and I have tried setting sriov_numvfs
to 7 in sysfs.conf
as well as trying to write it to the file directly, but it is not working. Which is fine if it is not intending to work yet, but I want to know what the correct way of doing this is.
I am on 6.11.0-rc6-1
I think "Newer platforms will be supported later" suggests it may be possible, but unlikely considering that support is not listed on intel's site: https://www.intel.com/content/www/us/en/support/articles/000093216/graphics/processor-graphics.html
But Tigerlake is also listed as unsupported there and listed as planned to be supported in the patch note I just linked, so who knows.
But if you want a dedicated card today that supports SRIOV I think there are some old AMD workstation cards that do that.
I'm still waiting for it to be mainlined cause I don't really feel like adding experimental patches to my kernel and its been like 2 years already
cause I don't really feel like adding experimental patches to my kernel
@adapt-L As far as I know, the Intel LTS kernel (the one this issue is in) supports SR-IOV with the i915 driver. If you don't want to patch your kernel or use the DKMS module that was extracted from this kernel (https://github.com/strongtz/i915-sriov-dkms), you could just run this kernel directly.
@Daniel15 Does the Intel XE LTS Kernel work with Proxmox and Intel Arc A770? I'd love to buy one if it's confirmed to work. If not, I might wait for Battlemage.
I guess we won't get an official Intel statement here, since Intel want to market there professional GPUs. I don't mind inofficial support anyways. :)
I honestly would find very strange even if unofficial support ends up happening.
@Daniel15 Does the Intel XE LTS Kernel work with Proxmox and Intel Arc A770?
I'm not sure as I haven't tried it. It should work if you copy the configuration across from the Proxmox kernel (so that all the same options are enabled). What I'm not sure about is if Proxmox apply any extra patches to their kernel, which would need to be applied too.
Hello!
The latest kernel tree from intel seems to work with SR-IOV on A770. It is based on the 6.11.0-rc2 kernel version, you might want to compile this manually.
You can find this kernel tree here:
https://gitlab.freedesktop.org/drm/xe/kernel
This is what i used for my configs:
/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on kvm.ignore_msrs=1 vfio_pci.ids=8086:56a0,8086:4f90 pcie_aspm=off"
/etc/modprobe.d/blacklist.conf
blacklist i915 blacklist xe blacklist snd_hda_intel
Using the Xe driver is the best option to have SR-IOV currently. i915 is not yet ready to be for SR-IOV. For new platforms it is good to switch from i915 to Xe for improved performance. Changing the driver can be done quite easily.
Thanks for the feedback. I might wait for Proxmox to pick up >= 6.12 once it's released and buy a A770 afterwards.
Using the Xe driver is the best option to have SR-IOV currently
Do you confirm that A770 supports SR-IOV with the Xe driver?
Currently no gpus are marked as has_sriov = true
So the SR-IOV support of the new xe driver just outputs "not supported" and then exits.
https://chatgpt.com/share/66fd8bbd-1434-8007-8e38-0b79ed59fdef
[ 14.379213] xe 0000:00:02.0: [drm] Support for SR-IOV is not available
[ 14.379215] xe 0000:00:02.0: [drm] Found ALDERLAKE_S/RPL-S (device ID a780) display version 12.00 stepping D0
[ 14.379216] xe 0000:00:02.0: [drm:xe_pci_probe [xe]] ALDERLAKE_S RPLS a780:0004 dgfx:0 gfx:Xe_LP (12.00) media:Xe_M (12.00) display:yes dma_m_s:39 tc:1 gscfi:0 cscfi:0
[ 14.379278] xe 0000:00:02.0: [drm:xe_pci_probe [xe]] Stepping = (G:D0, M:D0, B:**)
[ 14.379315] xe 0000:00:02.0: [drm:xe_pci_probe [xe]] SR-IOV support: no (mode: none)
[ 14.379349] xe 0000:00:02.0: [drm:intel_pch_type [xe]] Found Alder Lake PCH
Tested on Ubuntu Server liveCD 24.10 beta (kernel 6.11.0-7) I think the SR-IOV support is not ready yet.
- TGL (up to 7 VFs)
- ADL (up to 7 VFs)
- MTL (up to 7 VFs)
- ATSM (up to 31 VFs)
- PVC (up to 63 VFs)
Also it's interesting raptorlake-s is just a subplatform
of alderlake-s.
So if ADL is supported, RPL should also be supported.
static const struct xe_device_desc adl_s_desc = {
.graphics = &graphics_xelp,
.media = &media_xem,
PLATFORM(ALDERLAKE_S),
.has_display = true,
.has_llc = true,
.require_force_probe = true,
.subplatforms = (const struct xe_subplatform_desc[]) {
{ XE_SUBPLATFORM_ALDERLAKE_S_RPLS, "RPLS", adls_rpls_ids },
{},
},
};
static const struct pci_device_id pciidlist[] = {
...
XE_ADLS_IDS(INTEL_VGA_DEVICE, &adl_s_desc),
...
XE_RPLS_IDS(INTEL_VGA_DEVICE, &adl_s_desc),
...
};
Does that imply that all we need is a patch to change the has_sriov
flag to true
to get this to work?
The i915 DKMS SR-IOV driver works fine with Alderlake-S systems (I’m using one right now, with a i7-12700T.
I’m not sure hy it’s not enabled already. The 6.12 kernel is feature-frozen, isn’t it?
John T Davis
On Oct 2, 2024, at 1:28 PM, paulzzh @.***> wrote:
Currently no gpus are marked as has_sriov = true So the SR-IOV support of the new xe driver just outputs "not supported" and then exits.
https://chatgpt.com/share/66fd8bbd-1434-8007-8e38-0b79ed59fdef
[ 14.379213] xe 0000:00:02.0: [drm] Support for SR-IOV is not available [ 14.379215] xe 0000:00:02.0: [drm] Found ALDERLAKE_S/RPL-S (device ID a780) display version 12.00 stepping D0 [ 14.379216] xe 0000:00:02.0: [drm:xe_pci_probe [xe]] ALDERLAKE_S RPLS a780:0004 dgfx:0 gfx:Xe_LP (12.00) media:Xe_M (12.00) display:yes dma_m_s:39 tc:1 gscfi:0 cscfi:0 [ 14.379278] xe 0000:00:02.0: [drm:xe_pci_probe [xe]] Stepping = (G:D0, M:D0, B:**) [ 14.379315] xe 0000:00:02.0: [drm:xe_pci_probe [xe]] SR-IOV support: no (mode: none) [ 14.379349] xe 0000:00:02.0: [drm:intel_pch_type [xe]] Found Alder Lake PCH Tested on Ubuntu Server liveCD 24.10 beta (kernel 6.11.0-7) I think the SR-IOV support is not ready yet.
- TGL (up to 7 VFs)
- ADL (up to 7 VFs)
- MTL (up to 7 VFs)
- ATSM (up to 31 VFs)
- PVC (up to 63 VFs) Also it's interesting raptorlake-s is just a subplatform of alderlake-s. So if ADL is supported, RPL should also be supported.
static const struct xe_device_desc adl_s_desc = { .graphics = &graphics_xelp, .media = &media_xem, PLATFORM(ALDERLAKE_S), .has_display = true, .has_llc = true, .require_force_probe = true, .subplatforms = (const struct xe_subplatform_desc[]) { { XE_SUBPLATFORM_ALDERLAKE_S_RPLS, "RPLS", adls_rpls_ids }, {}, }, };
static const struct pci_device_id pciidlist[] = { ... XE_ADLS_IDS(INTEL_VGA_DEVICE, &adl_s_desc), ... XE_RPLS_IDS(INTEL_VGA_DEVICE, &adl_s_desc), ... }; — Reply to this email directly, view it on GitHub https://github.com/intel/linux-intel-lts/issues/33#issuecomment-2389418647, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGI5CYQY2P6QYFVCIQPGJ5TZZQ3L5AVCNFSM5Z63XG72U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMZYHE2DCOBWGQ3Q. You are receiving this because you commented.
Using the Xe driver is the best option to have SR-IOV currently
Do you confirm that A770 supports SR-IOV with the Xe driver?
SR-IOV not exist at A770 video card because GPU firmware(vBIOS) have this flag set at off. SR-IOV capabilities not supported on this card.
03:00.0 VGA compatible controller: Intel Corporation DG2 [Arc A770] (rev 08) (prog-if 00 [VGA controller])
Subsystem: Intel Corporation DG2 [Arc A770]
Flags: bus master, fast devsel, latency 0, IRQ 146, IOMMU group 14
Memory at 50000000 (64-bit, non-prefetchable) [size=16M]
Memory at 6000000000 (64-bit, prefetchable) [size=16G]
Expansion ROM at 51000000 [disabled] [size=2M]
Capabilities: [40] Vendor Specific Information: Len=0c <?>
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
Capabilities: [d0] Power Management version 3
Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
Capabilities: [420] Physical Resizable BAR
Capabilities: [400] Latency Tolerance Reporting
Kernel driver in use: i915
Kernel modules: i915, xe
https://lore.kernel.org/dri-devel/Zwekwrak12c5SSgo@fedora/
- Fixes and new development around SRIOV (Michal)
- More SRIOV work (Michal)
Let's wait for kernel 6.13
Let's wait for kernel 6.13
So, sometime around February 2025?
I know this is a technologically hard problem, and as an unannounced/alpha/skunkworks feature, we've really got no right to expect any sort of hard release date, but SR-IOV support on existing 11th/12th gen CPUs has been just around the corner since kernel 6.9 at least.
I've had great success with the StrongTZ DKMS SR-IOV driver, but using it involves high frequency updates/kernel rebuilds and complicates updating the OS itself (any time a new kernel is pushed, the DKMS driver has to be updated, and it might not be compatible (yet), so you'll have to wait, and if you're running Proxomx that applies to the hypervisor and all the guests).
(Aside: Anecdotally, 12th gen iGPUs seem to be the sweet spot for the SR-IOV DKMS driver. 13th gen works but seems a bit finicky, and 14th gen is a bit too new still for me to have a good feel for it yet.)
I'm really looking forward to this.
Kernel 6.12 feels really almost there, though, like we're just waiting for them to flip some switches to make certain CPUs actually show as having supported iGPUs.
Fingers crossed 6.13 gets over the line. :)
I've had great success with the StrongTZ DKMS SR-IOV driver, but using it involves high frequency updates/kernel rebuilds and complicates updating the OS itself
Not only that but it quickly adds up: I also need to maintain a working dkms for zfs by backporting compat patches from the development branch plus constantly rebasing bnx2x-2_5g dkms to be able to use my Broadcom at 2.5Gbps. You quickly start to hate DKMS at some point.
You quickly start to hate DKMS at some point.
@darkbasic at least it's better than the alternative - recompiling the whole kernel just to add or update one driver.
@jonas5
How to contact you, I want to know more about a770 sriov, my email is ccelkmi@gmail.com
Fingers crossed 6.13 gets over the line. :)
There's no has_sriov = true
in https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/gpu/drm/xe/xe_pci.c as of now :(
Fingers crossed 6.13 gets over the line. :)
There's no
has_sriov = true
in https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/gpu/drm/xe/xe_pci.c as of now :(
https://gitlab.freedesktop.org/drm/xe/kernel/-/commits/topic/xe-for-CI/?ref_type=heads https://gitlab.freedesktop.org/drm/xe/kernel/-/commit/5253d0e44a10fa23dfe1a610119e85a76e26397f They are testing in CI :) It's not ready yet.
Guess I would be getting myself an Intel based laptop after all then.
Since drivers for SR-IOV have already been completed in this fork, could these drivers be mainlined into the kernel?