Fullscreen sub-native resolutions cause checker artifacts in GNOME Wayland

[Whayme]

Hi, it's difficult finding exactly where to report this given that I've only been able to make this occur in GNOME's Wayland session but it seems to be a problem only with games run through Proton. Apologies if this should go somewhere upstream. I reported this to the Mutter issue tracker earlier here, if that's helpful.

When running a fullscreen game but at a resolution lower than the one my monitor is set to, I often get these black "checkers" all over the image. I haven't been able to replicate this in another compositor. They disappear when a system overlay like a volume meter is displayed over the game, and return when it fades. They don't show up in screenshots (so please forgive the photo of a television) and occur on many different displays. If I set the game to render at the native resolution of the display it's on, there is no problem. Well, to be clear it isn't the native resolution of the display per se but the current resolution of the desktop. For instance, a game running at 1280x720 will cause the artifacts on my 1440p monitor if I set it to 2560x1440, but not if I set the desktop to 1280x720. Also, the checkers disappear when using borderless fullscreen in games that support it, this seems to be an issue specifically with exclusive fullscreen modes.

I can reproduce the issue on both of my monitors as well as a TV. Every resolution I've tried results in the same thing on each as well, apart from the native ones. Regardless of aspect ratio, whether or not it scales evenly into the native res, or if it's very large / very small. I'm guessing it isn't a DXVK issue as I get checkers even in Doom 64 when using its native Vulkan mode.

Some games in which this occurs using Proton Experimental or 9.0-2: Doom 64, System Shock 2, Turok (2015), Elden Ring, Hitman: Contracts. Worth nothing that the Turok rerelease has checkers on the Windows version through Proton but not with the Linux-native one. I've just now learned that if I run Turok or System Shock 2 (the other two don't run on this old version in the first place) through Proton 5.0-10, there are no checkers, so it seems to have been a regression at some point.

steam-1148590.log Steam Runtime Diagnostics

[kisak-valve]

Hello @Whayme, please add PROTON_LOG=1 %command% to some affected game's launch options and attach the generated $HOME/steam-$APPID.log to this issue report as a file. (Proton logs compress well if needed.) Also, please copy the contents of Steam Runtime Diagnostics from Steam (Steam -> Help -> Steam Runtime Diagnostics) and put it in a gist, then include a link to the gist in this issue report.

This reads like a video driver issue that should be reported to your video driver vendor.

[Whayme]

I've added the requested info to the issue. It certainly could be a GPU driver issue but I imagine the fact that an old Proton version does not exhibit this behavior would get me sent right back here.

[matte-schwartz]

@Whayme This is very likely to be: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11294

and my related report which also ended up leading back to explicit sync with direct scan-out causing artifacting: https://gitlab.freedesktop.org/drm/amd/-/issues/3361

basically, direct scan-out with explicit sync on AMD seems to be an issue across the board at the moment, and your Steam Runtime Diagnostics fall under the criteria: Mesa 24.1.3-arch1.1 (newer than 24.1) Linux 6.9.9 (newer than 6.8)

[Whayme]

Interesting, though it's worth mentioning none of my issues show up through gamescope like yours do (or kwin for that matter).

[matte-schwartz]

Interesting, though it's worth mentioning none of my issues show up through gamescope like yours do (or kwin for that matter).

It's really situational depending on what apps/environments present the issue because of the delicate nature of getting direct scan-out to work, but it seems like many roads lead back to the same Mesa issue. That being said, it could also certainly be a different or new issue, but the conditions under which you're reporting artifacting + the Mutter issue report you created is very similar to the other direct scan-out issues I've seen.

[Whayme]

Would just like to add that while I claimed this doesn't occur in kwin and that used to be the case (I briefly used Plasma to get away from the bug), it doesn't seem to be now, as I'm getting it on 6.1.3.