Mass Effect Legendary Edition: a line across the screen is visible on some materials when Ambient Occlusion is enabled

[adolfintel]

In Mass Effect 1 Legendary Edition, a diagonal line going from the top left corner of the screen to the bottom right corner is visible on some materials when Ambient Occlusion is enabled. The line looks like a typical ambient occlusion shadow so it's visible enough to be annoying in movement but not enough to be very noticeable in screenshots.

Here's a magnified screenshot where the problem can be seen:

I wasn't able to solve this by reinstalling the game, with different versions of DXVK (including current master), or by changing any of the settings in the DXVK configuration file.

The only unusual thing I can see in the logs is this:

warn:  D3D11Query: Unknown interface query
warn:  9eb576dd-9f77-4d86-81aa-8bab5fe490e2

Video of the issue: ME1-video.mkv Apitrace file: ME1-trace.tar.zst DXVK log files: MassEffect1_d3d11.log MassEffect1_dxgi.log

Software information

Mass Effect 1 Legendary Edition

System information

• GPU: AMD Radeon RX 6900XT
• Driver: Mesa 21.3.7 (tested both RADV and AMDVLK)
• Wine version: Wine-GE-Proton7-8
• DXVK version: tested 1.8 through 1.10.1 incl. current master
• OS: Manjaro 21.2.5, KDE, Xorg, Kernel 5.16

[K0bin]

Are you sure this is actually a DXVK problem rather than just a game bug?

[adolfintel]

I don't see this line on Windows and I can't find anyone complaining about it online so I guess it's either DXVK or Mesa.

[adolfintel]

Looks like the HBAO shader source file is present in the game files in the Engine/Shaders folder, maybe I can do something to it...

[K0bin]

I can't reproduce the issue on my Nvidia GPU. Try a newer Mesa version.

[adolfintel]

Still present in Mesa 22.0.1 :(

[doitsujin]

Can't reproduce here (6900XT, both mesa 22.0 and current git). It also doesn't make much sense for this to happen since the AO texture itself is computed via a compute shader, and then blended onto a render target with regular linear filtering.

Do you have any options enabled to enforce anisotropic filtering or something?

[adolfintel]

Nope, the only clue I have is that it happens in 2560x1440 in fullscreen and borderless but not in windowed mode.

I'll try doing a clean install of the OS and let you know.

[adolfintel]

The problem still occurs with a clean install of the OS (tried both Xorg and Wayland just in case) but it works fine in Windows. WineD3D seems to be affected as well.

@doitsujin can you tell me how you're running the game? OS, screen resolution, etc.? The problem is not visible in every area of the game, I can give you a save file if you want.

[adolfintel]

Quick update on this, I just tested this on two more PCs (i5 6400+R9 290x, i7 6700k+RX 5500XT) with the same results, the line is present on linux but not on windows.

Are you 100% sure that you don't have this problem on your 6900XT? What else can I try to analyze this problem?

[Blisto91]

If you can share the save file i can try to see if i can reproduce it

[adolfintel]

Sure! BioWare.zip

Just load the last save in the citadel presidium and move around, you should see it.

[Blisto91]

Ah yes i see it. Easier to see in that spot when you turn stuff like film grain off. R9 380 mesa 22.0.1 and also on amdvlk 2021Q2.5 (latest i can use). The game or launcher crashes before it's starts when i try with WINED3D.

Didn't find that any dxvk conf options affected it.

[adolfintel]

@Blisto91 you can bypass the launcher and launch the game directly with this command:

MassEffect1.exe -NoHomeDir -SeekFreePCConsole

The line is visible with WineD3D as well on my PC.

Can you confirm that it goes away if you set it to windowed mode instead of fullscreen?

[Blisto91]

Ye can confirm windowed gets rid of the issue.

I can try to launch the game on my nvidia setup when i have time, and if it can handle it. Only 2GB vram hehe.

[CME42]

I can repro it on my 6900xt with both radv and amdvlk but not on my 1060, radv_disable_aniso_single_level=true doesn't help.

[Blisto91]

Thanks for testing on Nvidia. My 960 was screaming in pain just attempting to boot it 😁

I also tried various radv variables without a change.

[doitsujin]

Figured out what's wrong, D3D and Vulkan use different rules for computing texture coordinates when sampling with NEAREST filtering, and since the game uses a fullscreen quad to apply the AO effect, one triangle has tiny interpolation errors that lead to sampling offsets.

Unfortunately this is pretty much impossible to fix on our end and will probably need a driver-level workaround.

[Blisto91]

Good work. Followed the Convo on discord. 😁 So mesa issue and cross fingers? 👀

[adolfintel]

@doitsujin Thanks for investigating the issue.

Do you happen to know why I can't edit any of the .usf files? I want to try and modify the shaders to work around this issue but the game won't start if I change even a comment in those files.

(Also, can we remove the can't reproduce tag from this issue since we were able to reproduce it?)

[Herobrine24]

Do you happen to know why I can't edit any of the .usf files? I want to try and modify the shaders to work around this issue but the game won't start if I change even a comment in those files.

Legendary Edition does not come with a shader compiler. Thats why shaders arent editable.

[adolfintel]

Just checked again, the issue is still present as of june 2023 on the latest mesa master, was this reported to mesa? I can't find it in their issues

[Blisto91]

I don't believe so

[adolfintel]

Ok, opened an issue on mesa gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9253

[adolfintel]

Still here :(

I'm willing to put a bounty on this issue if the project allows it, I'm tired of having to reboot into windows every time I want to play this game, and this issue likely affects other late UE3 games that use the same ambient occlusion shader.

[mbriar]

I saw this issue mentioned here, have you tried the workaround posted there?

[adolfintel]

@mbriar that seems to have fixed it. The code was moved to radv_sampler.c but once I found it and patched it, the line was gone and AO looks correct.

I'm not sure what the implications of this change are though, maybe one of you veterans can explain: if this was only enabled when using the nearest filter, I assume there's a reason for that, what could this break? how do coordinates get truncated?

[Blisto91]

Fixed in mesa https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25911