[Bisected] Color corruption on intel graphics

[Samsagax]

Being using gamescope on my AMD machine for a while without issues. When trying out on my laptop Nvidia won't let me use it because of #151 but I managed to start gamescope on the intel GPU (iGPU) and then start games from it using the Nvidia GPU (dGPU) by setting some env variables (pretty cool, huh?).

Here is the issue: I started steam client within gamescope on my intel card by forcing VK_ICD_FILENAMES variable like so

$ VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/intel_icd.x86_64.json gamescope -- steam

The client launches fine but then the colors are corrupted:

But I managed to start a native game (selected Factorio, then Start) and then it all works inside the game (Launched with __NV_PRIME* variables for it to use the dGPU):

Also when the game is launching the window flickers back and forth with the corrupted colors client and the correct colored game. After the game is started, it all "just works".

[l3s2d]

Noticed this after upgrading to 3.10.3 on Arch Linux. Goes away after downgrading to 3.9.5.

[Samsagax]

Noticed this after upgrading to 3.10.3 on Arch Linux. Goes away after downgrading to 3.9.5.

Great. Never noticed this before but with this I'll try to bisect it.

[Samsagax]

Ok I managed to bisect it. Here is the log.

git bisect start
# good: [26aeebf11d2d195fd2fa75f0d85d11dcd3ee5e69] steamcompmgr: publish input change count
git bisect good 26aeebf11d2d195fd2fa75f0d85d11dcd3ee5e69
# bad: [dd22f8db76aa9726b64d7a21095d669aec3b3732] steamcompmgr: Fix validContents nullptr deref
git bisect bad dd22f8db76aa9726b64d7a21095d669aec3b3732
# good: [6402b5cd40c78e0972348133ffe1a57cedbb807c] steamcompmgr: Fix child window cursor barrier
git bisect good 6402b5cd40c78e0972348133ffe1a57cedbb807c
# good: [7e93200e084f49dc8b68b8d0f85ad15afccf2e9e] Dedupe and memoize fb_id and vulkan texture per buffer.
git bisect good 7e93200e084f49dc8b68b8d0f85ad15afccf2e9e
# bad: [296a3d498d02c1277e4de7ed2e67ad537b1d3553] steamcompmgr: Add framework for held commits
git bisect bad 296a3d498d02c1277e4de7ed2e67ad537b1d3553
# bad: [93e4d94a78c1d371f1e2112c6386fe3dfd085bb9] steamcompmgr: Add --cursor-hotspot
git bisect bad 93e4d94a78c1d371f1e2112c6386fe3dfd085bb9
# bad: [4ee56a1e75a043c6f317ea2f71c96b2061e85cf5] rendervulkan: Do not premultiply color
git bisect bad 4ee56a1e75a043c6f317ea2f71c96b2061e85cf5
# bad: [23430084e2245e226f27dbe1882bd30633a1d50c] rendervulkan: Blend in linear space
git bisect bad 23430084e2245e226f27dbe1882bd30633a1d50c
# first bad commit: [23430084e2245e226f27dbe1882bd30633a1d50c] rendervulkan: Blend in linear space

And the first bad commit:

23430084e2245e226f27dbe1882bd30633a1d50c is the first bad commit
commit 23430084e2245e226f27dbe1882bd30633a1d50c
Author: Joshua Ashton <joshua@froggi.es>
Date:   Sat Nov 13 22:28:25 2021 +0000

    rendervulkan: Blend in linear space

    We must blend in linear space, not SRGB space or we get these horrible browns.

    Gets us closer to how this looks with planes.

 src/composite.comp   | 28 ++++++++++++++++++++++++++--
 src/rendervulkan.cpp | 27 ++++++++++++++-------------
 2 files changed, 40 insertions(+), 15 deletions(-)

[misyltoad]

Once we fix validation stuff and add acquire barrier itd make sense to open an issue on Mesa if it isnt fixed.

[leonidx86]

Reporting my test on two different Intel iGPUs, maybe this will help to locate the issue. For both tests the Linux distro and package versions are the same:

• Kernel: 5.16.3.arch1-1
• Mesa: 21.3.5-1
• Gamescope: 3.11.0beta4-1

gamescope -W 320 -H 320 -m 1 glxgears

GPU: Intel(R) HD Graphics 4000 (IVB GT2)

GPU: Intel(R) UHD Graphics 630 (CFL GT2)

[Samsagax]

Might worth a try with mesa-git package to see if it's fixed upstream.

[leonidx86]

Might worth a try with mesa-git package to see if it's fixed upstream.

Tested now, same result with mesa-git (version 22.0.0_devel.149533.24fef8f33da.d41d8cd98f00b204e9800998ecf8427e-1).

[DadSchoorse]

This should be reported to mesa, as far as I know we do everything correctly now. All the validation errors have been fixed.

[vignedev]

Not sure how helpful this might be, but I've been experimenting/hacking a few changes to try to get it visually working on my setup.

There are two scenarios that I have tested, both involving changes in the s_DRMVKFormatTable structure and both resulting in a non-glitch state, however with different bugs:

---

1. Forcing both vkFormat and vkFormatSrgb to be their sRGB variant

   struct {
   uint32_t DRMFormat;
   VkFormat vkFormat;
   VkFormat vkFormatSrgb;
   bool bHasAlpha;
   } s_DRMVKFormatTable[] = {
   -       { DRM_FORMAT_ARGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_SRGB, true },
   -       { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_SRGB, false },
   +       { DRM_FORMAT_ARGB8888, VK_FORMAT_B8G8R8A8_SRGB, VK_FORMAT_B8G8R8A8_SRGB, true },
   +       { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_SRGB, VK_FORMAT_B8G8R8A8_SRGB, false },
   { DRM_FORMAT_NV12, VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, false },
   { DRM_FORMAT_INVALID, VK_FORMAT_UNDEFINED, VK_FORMAT_UNDEFINED, false },
   };

   This makes gamescope run with correct colors, however enabling FSR results in black screen.

Without FSR	With FSR

---

2. Forcing both vkFormat and vkFormatSrgb to be their UNNORM variant

struct {
    uint32_t DRMFormat;
    VkFormat vkFormat;
    VkFormat vkFormatSrgb;
    bool bHasAlpha;
} s_DRMVKFormatTable[] = {
-       { DRM_FORMAT_ARGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_SRGB, true },
-       { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_SRGB, false },
+       { DRM_FORMAT_ARGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_UNORM, true },
+       { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_UNORM, false },

    { DRM_FORMAT_NV12, VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, false },
    { DRM_FORMAT_INVALID, VK_FORMAT_UNDEFINED, VK_FORMAT_UNDEFINED, false },
};

This makes gamescope run in incorrect washed-out colors (obviously due to color conversion), however when using FSR all colors return into their properness. This however required the revertion of changes brought by eff99fdf813c199e360d73b0b5a626f048f7f259 to get it working.

Without FSR	With FSR

I'm not experienced at all in these regions and I found these changes purely by trial and error, so I am unsure of any possible side-effects nor the full purpose of such changes, however I hope it will help narrow the issue down somewhat.

[patrickaldis]

I can confirm I'm having near identical same issues on an 7700HQ under intel UHD 630

Created an issue here, but realising this is actually the same as this thread

https://github.com/Plagman/gamescope/issues/393

[DadSchoorse]

@vignedev All of that indicates that it's a driver issue, so something that should be reported to mesa.

[patrickaldis]

@DadSchoorse Would this really be a mesa issue if the game runs without gamescope on mesa?

[DadSchoorse]

@patrickaldis There's always the possibility that there's some subtle bug in our code, but given that gamescope works fine on radv I think it's an anv problem. The hacks by @vignedev show that there's is some issue with mutable images + dmabufs and with the import barriers. The game doesn't matter.

[leonidx86]

As an anecdotal evidence, I've tested this on several different Intel CPUs with integrated graphics. Only one of them (UHD 630) has this visual artifact (but works fine without Gamescope). Everything else works fine with or without Gamescope.

@Samsagax Which Intel CPU/iGPU have you tested on? @l3s2d Says the issue is not present on v3.9.5, but I can't downgrade, since there is no package below v3.11.0 at https://archive.archlinux.org/packages/g/gamescope/.

[ccajas]

I'd like to add my two cents in, and say that these problems occur on most games I've tried, but there are at least a few games where Gamescope is mostly working fine with Intel UHD graphics. I'm running it on a Celeron N4120 with UHD 600 graphics.

I've tested out Portal and Lara Croft: Temple of Osiris and both these games are playable with Gamescope, at least in all the sections with real-time 3D graphics.

Portal is very playable, only seeing image artifacts in the Valve splash screen and the death sequences. Temple of Osiris starts with a setup window where artifacts show up, but once you get past that window it all looks fine.

[Samsagax]

As an anecdotal evidence, I've tested this on several different Intel CPUs with integrated graphics. Only one of them (UHD 630) has this visual artifact (but works fine without Gamescope). Everything else works fine with or without Gamescope.

@Samsagax Which Intel CPU/iGPU have you tested on? @l3s2d Says the issue is not present on v3.9.5, but I can't downgrade, since there is no package below v3.11.0 at https://archive.archlinux.org/packages/g/gamescope/.

I have an i5 6200U cpu, the iGPU is a 520 (SKL GT2).

[Samsagax]

For what is worth, seems to be a problem with the vulkan bit of the intel driver for those iGPUs. Games that use OpenGL will not have this issue.

Does anyone have an account on the mesa gitlab instance to report this? I think we colected valuable information and should be able to report it to be fixed easily.

@l3s2d Says the issue is not present on v3.9.5, but I can't downgrade, since there is no package below v3.11.0 at https://archive.archlinux.org/packages/g/gamescope/.

You would need to use git tags for this. Use the gamescope-git package as a reference.

[Oschowa]

I can reproduce this issue on my Kaby Lake UHD 620 using Mesa 21.3.5. However, on gamescope master and latest Mesa main it's working fine. INTEL_DEBUG=norbc also makes it work fine on Mesa 21.3.5 (it will however likely reduce performance).
This Mesa commit that fixes the issue is https://gitlab.freedesktop.org/mesa/mesa/-/commit/ca791f5c5d88a6451e4934f3e03fa0e38a7759fa

[leonidx86]

@Oschowa You're right! With INTEL_DEBUG=norbc it works fine. However, with latest Mesa (22.1.0) and relatively latest Gamescope (3.11.0beta6-1) the issue is still there on UHD 630.

[Oschowa]

Yes, it seems like latest Mesa only fixes it for vulkan apps running in gamescope, anything using OpenGL is still broken. I opened a Mesa issue about this here https://gitlab.freedesktop.org/mesa/mesa/-/issues/6029

[JordanViknar]

I can confirm I also encounter this issue running Minecraft Java Edition or glxgears on my Intel HD Graphics 500 iGPU.

[patrickaldis]

The artifacts in my case seem to be more extreme:

[adolfintel]

I also have this problem on Intel UHD 600 (Gemini Lake)

[Samsagax]

I'm currently testing some stuff and the issue seems to be fixed. Could anyone please confirm? Relevant package versions:

• linux-zen 5.16.16.zen1-1
• mesa 21.3.7-2
• vulkan-intel 21.3.7-2
• vulkan-icd-loader 1.3.208-1

[leonidx86]

@Samsagax Nope. Even installed the Zen Kernel. The only major difference with your config seems to be the GPU.

• GPU Intel(R) UHD Graphics 630 (CFL GT2)
• linux 5.16.16-arch1-1 and 5.16.16-zen1-1-zen
• mesa 21.3.7-2
• vulkan-intel 21.3.7-2
• vulkan-icd-loader 1.3.208-1

[JordanViknar]

I'm currently testing some stuff and the issue seems to be fixed. Could anyone please confirm? Relevant package versions:

• linux-zen 5.16.16.zen1-1
• mesa 21.3.7-2
• vulkan-intel 21.3.7-2
• vulkan-icd-loader 1.3.208-1

The issue still occurs for me, using the latest Arch Linux packages.

[JordanViknar]

Just noticed, but the issue also happens on my other laptop (which doesn't really need it, unlike my main one) still with the latest packages, but with this Intel card :

• Intel CoffeeLake-H GT2 [UHD Graphics 630]

That laptop ALSO has an NVIDIA card, and I used the same workaround. Doesn't seem to be related though.

[shmugoh]

Intel HD Graphics 620 iGPU is also affected by this issue. A bit unrelated but might aswell say it. This pops up at the start

[Samsagax]

The CAP_SYS_NICE is unrelated for sure. But the other option might not be. I'll test it and report back.

[leonidx86]

The sysctl setting is also unrelated as far as I know. echo "dev.i915.perf_stream_paranoid = 0" > /etc/sysctl.d/60-mdapi.conf and reboot. More info: https://www.intel.com/content/www/us/en/develop/documentation/vtune-help/top/installation/set-up-system-for-gpu-analysis.html

[shmugoh]

Gave it a try with mesa-git-22.1.0_devel.151862.44aff2beec8.d41d8cd98f00b204e9800998ecf8427e-1 cause I thought it would be a mesa problem, and still the same result; might be a gamescope issue I guess.

The CAP_SYS_NICE is unrelated for sure. But the other option might not be. I'll test it and report back.

I also tried the sysctl option aaand still the same result.

[shmugoh]

Just managed to get gamescope working!.. by using an older version (3.9.5 to be specific)

If anyone wants to downgrade to 3.9.5, uninstall gamescope first and run these commands:

git clone https://github.com/Plagman/gamescope.git
cd gamescope
git reset --hard 26aeebf11d2d195fd2fa75f0d85d11dcd3ee5e69 # 3.9.5 commit SHA1

# Build
git submodule update --init
meson build/
ninja -C build/

# Install
meson install -C build/ --skip-subprojects

I know this has already been mentioned before but I'll leave the commands here anyway just in case if theres a lazy person trying to set this up.

[shmugoh]

For those who don't wanna downgrade gamescope, use INTEL_DEBUG=norbc. I'm still sticking to 3.9.5 though, as the latest version is a bit laggy and audio stutters a lot, but that might be different for you if you have a stronger PC.

[Samsagax]

For those who don't wanna downgrade gamescope, use INTEL_DEBUG=norbc. I'm still sticking to 3.9.5 though, as the latest version is a bit laggy and audio stutters a lot, but that might be different for you if you have a stronger PC.

I can confirm with latest gamescope this environment variable "fixes" it. At least the color corruption dissapears.

[boykokanev]

Same color corruption, INTEL_DEBUG=norbc fixes it.

inxi -G Graphics: Device-1: Intel Skylake GT2 [HD Graphics 520] driver: i915 v: kernel Device-2: Microdia Integrated_Webcam_HD type: USB driver: uvcvideo Display: x11 server: X.Org v: 1.21.1.3 with: Xwayland v: 22.1.1 driver: X: loaded: modesetting,radeon gpu: i915 resolution: 1920x1080~60Hz OpenGL: renderer: Mesa Intel HD Graphics 520 (SKL GT2) v: 4.6 Mesa 21.3.8

[ghost]

I'm here from https://github.com/Plagman/gamescope/issues/480. I was seeing horrible distortions with gamescope similar to the images in this ticket. (Intel Iris Plus G7 with Mesa 21.3.8 and gamescope 3.11.22)

I can confirm that INTEL_DEBUG=norbc fixes my issue.

[xiota]

I also have this corruption issue with Mesa 22.0.1, 22.1.4, and a git build on an Intel Skylake GT2 [HD Graphics 520]. It seems to occur with anything that doesn't use Vulkan. For some games, Mesa can be forced to translate OpenGL to Vulkan with MESA_LOADER_DRIVER_OVERRIDE=zink, but it has to be done in a roundabout way:

gamescope -- sh -c 'MESA_LOADER_DRIVER_OVERRIDE=zink glxgears'

If desired, the same method can be used to limit the effect of INTEL_DEBUG=noccs (norbc is deprecated) to only the game.

I don't know whether Zink or noccs performs better. noccs works on non-OpenGL games.

Notes:

• I've since discovered that the noccs/norbc workaround does not work for all programs. In particular, games run in DOSBox have corrupted graphics when run with nocss, but Zink seems to prevent it.
• Neither Zink nor `noccs prevents graphics corruption in DOSBox-X.

[tyisafk]

Same issue on Intel Arc A750. The INTEL_DEBUG=norbc (and noccs) variable fixes it.

[luni3359]

I only came across this bug after running 250900 (including all the DLC) with --prefer-vk-device 8086:9bc4 (igpu). The game refuses to boot if I don't choose to prefer intel, for some reason nvidia doesn't work so I have to force it.

The game looks fine with the intel debug fix. I'm not sure if it's related but the game takes a while to start, it hangs on a black screen for like 10 seconds and I have an ssd, running on switchable graphics.

Specs:

CPU: Intel Core i5-10300H (8) @ 4.5 GHz GPU 1: Intel UHD Graphics GPU 2: NVIDIA GeForce GTX 1660 Ti Mobile

Please let me know what other information I should provide if needed.

[Schlaefer]

Run into the same issue on an Intel Arc 750.

While testing things out I noticed that everything looks OK when also starting mangohud via env variable. So I just moved the HUD off screen:

env MANGOHUD_CONFIG=offset_x=-500 MANGOHUD=1 gamescope -h 1080 -H 2060 -f -- %command%

It's ugly, but so far seems to work fine.

Graphics:
Device-1: Intel DG2 [Arc A750] driver: i915 v: kernel
Display: wayland server: X.org v: 1.21.1.6 with: Xwayland v: 22.1.7
compositor: kwin_wayland driver: N/A resolution: 1: 2194x1234 2: 1920x1080
3: 1920x1080
API: OpenGL v: 4.6 Mesa 23.0.0-devel (git-1af39eda6e) renderer: Mesa
Intel Arc A750 Graphics (DG2)

[amindWalker]

The problem seems to be related to the exclusive fullscreen mode and only happens with Intel but not with Nvidia driver (Nvidia has a lot of other bugs tho).

The only way to get it to work correctly is to set INTEL_DEBUG=norbc and using windowed with almost fullscreen or by using my workaround to fullscreen:

• Have the top bar (I use GNOME) to be always on top of everything else. I'm using a GNOME extension called Dash to Panel with Intellihide. It's possible to test this issue by simply using the default extension shortcut SUPER+i to show/hide the top bar. The color corruption appears immediately when the top bar disappears.

Hardware tested:

• Arch Linux with 6.2 Kernel
• MESA 23.1 master git
• Intel 9750H with 630 UHD iGPU
• RTX 2060

[BobrImperator]

I've been playing around with the noccs norbc INTEL_DEBUG settings, and noticed that while https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15447 mentions noccs to be an alias of norbc it gives me different results when booting up Star Citizen via lutris.

Thought I'd document that behaviour here.

INTEL_DEBUG=noccs

INTEL_DEBUG=norbc

glxinfo -B

name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Intel (0x8086)
    Device: Mesa Intel(R) Arc(tm) A770 Graphics (DG2) (0x56a0)
    Version: 23.0.2
    Accelerated: yes
    Video memory: 16288MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.6
    Max compat profile version: 4.6
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
OpenGL vendor string: Intel
OpenGL renderer string: Mesa Intel(R) Arc(tm) A770 Graphics (DG2)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 23.0.2
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.6 (Compatibility Profile) Mesa 23.0.2
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.2 Mesa 23.0.2
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

cc original issue where I've found about these flags https://github.com/ValveSoftware/gamescope/issues/836#issuecomment-1519230500

[ccallawa-intel]

It'd be useful to know how many of these issues are resolved by https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22656/

[ruineka]

It'd be useful to know how many of these issues are resolved by https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22656/

I'll test it out on my intel devices and come back with results.

[ruineka]

It'd be useful to know how many of these issues are resolved by https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22656/

Iris Xe typically has a black screen when the mouse is moved, but we patch ChimeraOS to work around this problem and previously we had to use INTEL_DEBUG=norbc to fix the color corruption we'd have after fixing the black screen issue. https://github.com/Samsagax/gamescope/commit/6f668ee8b9dde03fea8c5581780e86f4e2729df3

Testing using ChimeraOS which uses gamescope with the gamepadui matching the Steam Deck experience.

• Intel A750 tested (Corruption when cursor is moved without INTEL_DEBUG=norbc
• Intel 1195G7 (No corruption when cursor is moved, blackscreen issue remains. I'm not sure if the patch was what fixed this or not. I will need to try mesa-git later when I have time.)
• Intel UHD 630 (Not tested, I'm not entirely sure the iris driver works for this)

[paperlyly]

I have the same problem, but when using INTEL_DEBUG=norbc the games do not open, I use an Intel Skylake GT2 [HD Graphics 520]

[ccallawa-intel]

Another Mesa PR that may help here: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22988

As before, testing appreciated.

[paperlyly]

tried with both INTEL_DEBUG=norbc and INTEL_DEBUG=noccs

the colors return to normal, the problem is that a lot of performance is lost, any advice????

[sqrtd]

Another Mesa PR that may help here: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22988

As before, testing appreciated.

Not sure if that was the MR that fixed it, but after installing the latest mesa-git (after the MR was merged) I don't see any corruption on my system. I'll test it again later today to make sure.

[paperlyly]

Another Mesa PR that may help here: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22988 As before, testing appreciated.

Not sure if that was the MR that fixed it, but after installing the latest mesa-git (after the MR was merged) I don't see any corruption on my system. I'll test it again later today to make sure.

can be solved by installing mesa-git???

can it be installed on debian?