"Encoder: Image creation failure: Unknown error" after update to Mesa 23.3.6

[manfredg]

Description

Everything worked amazingly well on Mesa 23.2.1 but after I updated to the latest Mesa 23.3.6 I am getting the error "Encoder: Image creation failure: Unknown error" followed by "error in encoder thread: Failed to initialize vulkan frame context: Invalid argument"

I managed to trace the problem to a failed call to av_hwframe_ctx_init() in ffmpeg_helper.cpp which in turn calls vulkan_frames_init() in ffmpeg's hwcontext.c itself calling create_frame() on line 2374 in hwcontext_vulkan.c

The call seems to be the exact same with identical parameters but for some reason the new Mesa 23.3.6 refuses to create this frame and returns an unknown error.

I can regain full function by swapping to the original libvulkan_intel.so library version 23.2.1, no other changes to ICD files or any other part of Mesa, so the problem seems to be with this exact function.

I realise that this is technically on the ffmpeg side of things and I did try to change the dependency in the ALVR sources from ffmpeg 6.0 to 6.1.1 but because of various patches being applied this didn't work out.

General Troubleshooting

• [x] I carefully followed the instructions in the README and successfully completed the setup wizard
• [x] I read the ALVR Wikis here and here

Environment

Hardware

hw-probe -all -upload: https://linux-hardware.org/?probe=0ee555fcd8

Installation

ALVR Version:

Latest git, but problem has already persisted for weeks.

ALVR Settings File:

session.json

SteamVR Version: 2.3.5

Install Type:

• [ ] Packaged (exe, deb, rpm, etc)
• [ ] Portable (zip)
• [x] Source

OS Name and Version

openSUSE Tumbleweed

[manfredg]

@nowrep, apologies for calling you out, but I believe you last worked on the Linux ffmpeg code. Do you have any insight why the latest Mesa libvulkan_intel.so library doesn't accept the arguments anymore it receives from ffmpeg?

[nowrep]

Mesa 23.6.3

This is not a valid version. (23.2 -> 23.3 -> 24.0 -> ...)

I managed to trace the problem to a failed call to av_hwframe_ctx_init() in ffmpeg_helper.cpp which in turn calls vulkan_frames_init() in ffmpeg's hwcontext.c itself calling create_frame() on line 2374 in hwcontext_vulkan.c

Sounds like ffmpeg bug or mesa regression.

But also the Vulkan ffmpeg context in alvr is only used for nvidia, so probably it could just ignore these errors on intel/amd.

[manfredg]

Thanks for the quick reply and apologies for the typo, it should have been 23.3.6 rather than 23.6.3. I can see how that was very unhelpful in identifying the issue.

If the respective ffmpeg code is not required for intel/amd hardware, perhaps it could be factored out rather than let it fail? Not sure why it would fail though.

[manfredg]

For what it's worth, I skipped the call to av_hwframe_ctx_init() in ffmpeg_helper.cpp by commenting it out and this solves the issue for me.

Maybe if one calls cargo prepare-deps with the --no-nvidia option this part of the code shouldn't be compiled in, if it's only used on nvidia cards?

Or maybe somebody can find out why this fails in the first place. It used to work just fine until Mesa 23.2.1 but hasn't worked since. The latest version I tried was Mesa 24.0.3. Possibly it should fail because it's nvidia specific, but I haven't got the insight to understand what this call is really doing, which to me looks like generic Vulkan code.

[YannHS]

I can confirm that this happens on an Intel ARC a750 as well. Some games don't render correctly on the older mesa versions and require a broken version to render correctly so there is definitely some benefit to having this work over just using the old version of mesa.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[manfredg]

To the best of my knowledge this still hasn't been fixed even though the fix is trivial. Just don't call av_hwframe_ctx_init() on Intel graphics cards.

[ProxySocks]

Has this been resolved by anyone yet? The issue is still present for me. Arc A770, newest version of ALVR and SteamVR.

[Akeem29]

I'm having the exact same issue with GTX 1650. Mesa version is 24.1.5 and Nvidia driver version is 555.58.02. Instead of SteamVR home I get black screen in the VR

[Alvenster]

Same issue on Debian 12 Stable. Dell G7 with Nvidia RTX 2060. Can provide more information if necessary. A fix is greatly appreciated. I need this for my studies.

[YannHS]

If you really need VR streaming maybe try Wivrn. I've found it to be much easier to work with since it is standalone from SteamVR. https://lvra.gitlab.io has a trove of information about this.

[The-personified-devil]

I'm having the exact same issue with GTX 1650. Mesa version is 24.1.5 and Nvidia driver version is 555.58.02. Instead of SteamVR home I get black screen in the VR

Afaik this is an intel specific issue. Are you actually getting the error mentioned or just the black screen issue? If it's the latter you should go check out https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting, it has a fix for that.

[Akeem29]

I'm having the exact same issue with GTX 1650. Mesa version is 24.1.5 and Nvidia driver version is 555.58.02. Instead of SteamVR home I get black screen in the VR

Afaik this is an intel specific issue. Are you actually getting the error mentioned or just the black screen issue? If it's the latter you should go check out https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting, it has a fix for that.

I am having the error mentioned. This may have something to do with the fact that my PC has an integrated Intel GPU (Intel® UHD Graphics 730 (ADL-S GT1) in the 12th Gen Intel® Core™ i5-12400 CPU) that I can't disable because my monitor could only be connected to the motherboard's HDMI port (the GPU's HDMI port is inaccessible in my PC case). But I wrote __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia %command% in the SteamVR's launch options. With or without them the two mentioned error windows (one with "Encoder: Image creation failure: Unknown error" and the other one with "error in encoder thread: Failed to initialize vulkan frame context: Invalid argument") aren't affected

[The-personified-devil]

@Alvenster @Akeem29 you both seem to suffering from the integrated gpu taking precedence rather than the dedicated gpu. Note that you currently always need to apply the first fix on the linux troubleshooting page. In your case you'd also need the nvidia dedicated gpu fix. If that doesn't work you should run vulkaninfo (normally available through vulkan-tools) with the environment variables from the dgpu fix and report back what it outputs to debug the gpu selection.

Although since what you're encountering aren't really bugs but rather configuration issues and they're in fact unrelated to this issue you'd probably be better off asking for help on the discord/matrix.

[Meister1593]

Couple of fresh notes to the issue: I got it to work only on mesa-tkg-git (https://github.com/Frogging-Family/mesa-git/tree/master) and with https://github.com/Aikufurr/ALVR/commit/f1ffa2a94e72426a0eeb6940be76a3a948fb6dfc commit Intel Arc igpu on laptop

software encoder though doesn't work, it crashes steamvr, and backtracing points to this only (no idea what's wrong, segfault happens here, variables are present and it's not out of boundary)