Risen 3 - broken visuals

[aufkrawall]

The game's visuals are severely broken, while it works with wined3d:

Here is an apitrace: https://drive.google.com/open?id=18L9br-WvuI97-N4inOqYtM-8InYMbw7j

[axeldavy]

The game seems to try to use a non depth buffer format as depth buffer (it crashes on mesa with asserts). I'll investigate what should be done in this case.

[axeldavy]

Please try the game with this patch: https://github.com/iXit/Mesa-3D/commit/590ceb8dfdd191d1699d4a4d7e81b27fb56cdfff (if you want to apply to your build: https://github.com/iXit/Mesa-3D/commit/590ceb8dfdd191d1699d4a4d7e81b27fb56cdfff.patch)

If that doesn't help, you can also try launch the game with dynamic_texture_workaround=true

If none helped, could you make an apitrace under wined3d ?

[aufkrawall]

I applied the patch on top of mesa-master, but it doesn't help (dynamic_texture_workaround=true neither). So here is an apitrace for wined3d: https://drive.google.com/open?id=17EjfprQiIdgt0lhEIDu8LUv6OJLhB2xD

[aufkrawall]

In D9VK, Risen 2/3 and Gothic 3 need to have readonly locking disabled: https://github.com/Joshua-Ashton/d9vk/commit/efd98723969a6e0333e0c26f1af68c57e8313202 Uneducated guess: Perhaps there is something similar required for Nine?

[aufkrawall]

Instantly crashes after short corruption with RX 5700 XT and mesa-git of today.

[axeldavy]

Is it the only game suffering from corruption and crash ?

[aufkrawall]

Yep. Afaict, Gallium Nine generally works fine here in other titles. Games actually run much better than with AMD's native D3D9 driver for Navi. :)

[axeldavy]

with mesa git and rx480, the traces seem to behave as before

[axeldavy]

When you replay your wine trace, does it crashes for you ?

[aufkrawall]

I unfortunately don't have access to my wined3d trace anymore. But when I replay a new trace captured with native D3D9, Gallium Nine (the apitrace Wine process) crashes, whereas DXVK does not. Though there even is corruption when replaying the trace on Windows with native D3D9 (unless apitrace replay does some API wrapping which I'm not aware of), but it doesn't crash.

[axeldavy]

Strange for the crash. Could I get this new trace ?

Also I'm not surprised for the artifacts. The wine trace has them too. I think the app does something illegal when locking buffers and apitrace doesn't record it.

[aufkrawall]

Native D3D9 trace in 720p 10fps: https://drive.google.com/file/d/1JJnVuUFWgx53DObZYXZT7XfOFmW1jfIj/view?usp=sharing Haven't checked yet whether it crashes Nine, but the 1440p trace with unlocked fps did after a few seconds.

[axeldavy]

The artifacts are due to an apitrace bug when the trace is made, as shown here:

apitrace seems to have trouble tracking double lock.

EDIT: if you have made the trace with last release of apitrace, please report them the bug.

[axeldavy]

I think we also have a problem with double locks which cause the crash, I'll investigate. However apitrace has a bug too which causes the artifacts.

[axeldavy]

I tried this patch https://github.com/iXit/Mesa-3D/commit/cd60dbe4c8edd6439e6855fe4cf3b20d60a4f51e But it doesn't seem to help with the crash of the trace, I'll have to investigate further. It might be possible though that it helps the game, would it be possible for you to try it ?

[aufkrawall]

Sure, I'll also report the apitrace issue to the devs.

[aufkrawall]

I tried this patch iXit/Mesa-3D@cd60dbe But it doesn't seem to help with the crash of the trace, I'll have to investigate further. It might be possible though that it helps the game, would it be possible for you to try it ?

It still crashes with heavy corruption.

[axeldavy]

Thanks for the test, I realize now why the patch is wrong. If two threads do lock the same buffer at the same time, when you receive one unlock you have no way to know which lock it corresponds to. The true one depends on which thread wrote faster. Thus I understand now that the behaviour of wine and dxvk, which process the unmap only if all unlock were received is the correct one.

[axeldavy]

I made a new patch for the new behaviour.

Again the trace still crashes, but maybe the game is better ? https://github.com/iXit/Mesa-3D/commit/cd6f02b363b7c97a182c5ad30e1401a6440bad50

[aufkrawall]

Crashing/corruption is unchanged. :(

Btw.: Reported the apitrace issue, as the reference above reveals.

[axeldavy]

The apitrace doesn't crash if I disable an alternate optimized path for buffer allocation. I'm investigating...

[axeldavy]

Quite frankly I don't understand why it is crashing with the optimized path. Apitrace seems to indicate it crashes trying to write to a pointer that was released long ago. But I see no indication that this pointer was wrongfully released. Maybe without the optimized path, the pointer would have been reused, and thus would have been valid. I suspect an apitrace issue related to what was reported. But then it doesn't explain why the game crashes too.

[axeldavy]

This patch disables the optimized path: https://github.com/iXit/Mesa-3D/commit/828d6774a816ec3c94ae4c7d2f07a5b2a629184c

It makes the trace run fine here. Is the game affected ?

[aufkrawall]

I'm not able to apply it to latest git, or do I need the previous patch as well? patching file src/gallium/frontends/nine/buffer9.c Hunk #1 FAILED at 368. 1 out of 1 hunk FAILED -- saving rejects to file src/gallium/frontends/nine/buffer9.c.rej patching file src/gallium/frontends/nine/device9.c Hunk #1 succeeded at 322 (offset -3 lines). patching file src/gallium/frontends/nine/nine_buffer_upload.c

[axeldavy]

Only this line is needed: This->buffer_upload = NULL;//nine_upload_create(This->pipe_secondary, 4 1024 1024, 4);

[axeldavy]

Also, just in case, make sure that the mesa you update is 32 bits if the game is 32 bits (and 64 bits if it is 64 bits). Maybe you are updating the wrong mesa (because I would really have suspected the previous time the patch to fix it).

[aufkrawall]

Alright, building right now.

Yes, your remark regarding 32 bits is very valid, as the original game was 32 bits only, whereas it got updated to 64 bits only. Now there is only one executable, which is 64 bits. Thus I've always patched and built Mesa 64 bits.

[aufkrawall]

The apitrace capture now doesn't crash anymore, whereas the game still does (I've also checked that it loads everything from 64 bits /usr/lib path). :(

Edit: Just in case, how the patch stub looks:

diff --git a/src/gallium/frontends/nine/device9.c b/src/gallium/frontends/nine/device9.c
index 11294e562ba..2065ff6edac 100644
--- a/src/gallium/frontends/nine/device9.c
+++ b/src/gallium/frontends/nine/device9.c
@@ -325,7 +325,7 @@ NineDevice9_ctor( struct NineDevice9 *This,

     This->workarounds.dynamic_texture_workaround = pCTX->dynamic_texture_workaround;

-    This->buffer_upload = nine_upload_create(This->pipe_secondary, 4 * 1024 * 1024, 4);
+    This->buffer_upload = NULL;//nine_upload_create(This->pipe_secondary, 4 * 1024 * 1024, 4);

     /* Initialize a dummy VBO to be used when a vertex declaration does not
      * specify all the inputs needed by vertex shader, on win default behavior

[axeldavy]

I see... Well I see two potential answers: . Either there is another problem (like for the depth buffer issue) related to flags/formats advertised, and the game is taking a different path for gallium nine, which fails. . Either the game is assuming a specific pitch for some texture (I see apitrace complaining about pitch being different to what is recorded for some textures).

I would assume the first solution.

Could you see what were the last calls made before the crash by running with NINE_DEBUG=all csmt_force=0

Mesa needs to be built with debug or -D b_ndebug=false (I advise to use the latter for speed).

[aufkrawall]

Yes, I likely can do that by tomorrow evening. I'll also try to get Risen 1/2 and Gothic 3 by the same developer running in Wine to check whether they might show nasty behavior as well.

[axeldavy]

Great, here is what I need: . Mesa compiled with -D b_ndebug=false, and patch https://github.com/iXit/Mesa-3D/commit/cd6f02b363b7c97a182c5ad30e1401a6440bad50 . NINE_DEBUG=all csmt_force=0 wine YOUREXE (for example steam) &> a_file . Compress the file and upload it

[aufkrawall]

Here we go: a_file.zip (It's a 7-Zip archive renamed to zip in order to allow it as attachment.)

[axeldavy]

csmt_force=0 doesn't seem to have worked, was it typed correctly ?

The end of the trace is a bit weird (last function is a function it called already with same arguments before, so I don't see why it would freeze/crash). Could you run get a new log to confirm it is crashing here (and not in another thread due to another function) ?

[aufkrawall]

.bash_history lists NINE_DEBUG=all csmt_force=0 wine '/home/me/.wine/drive_c/Program Files (x86)/Steam/steam.exe' &> a_file. Does it matter that I have generally set WINEDEBUG=-all? I assumed it would affect only Wine, not Nine. I closed Steam regularly after the game had crashed. Would it be better if I ran wineserver -k instead?

Edit: Hm, perhaps it would work if I globally exported csmt_force=0, in case it wasn't passed down to every child process? I had a similar issue once.

[aufkrawall]

Hm, env lists csmt_force=0, but new log still contains ...CSMT worker spawned: a_file.zip

[axeldavy]

Thanks for the new log. I don't understand why csmt_force=0 is not working. The issue is that two threads can write at the same time in the log which gives some garbage. Anyway for the two logs, it shows it doesn't crash at the same location, but in (or after) the same function.

Could you get a new log adding patch https://github.com/iXit/Mesa-3D/commit/5a1a902f3650b6e25a84fc56b64a09fe56b5ebb8

[aufkrawall]

Sure. I tried Risen 2, 1 and Gothic 3: Risen 2 seems to work without issues, whereas Risen 1 crashes after loading a savegame and Gothic 3 freezes. They work with DXVK.

Edit: Does the new patch replace the old one?

[axeldavy]

Thanks or the tests, sad to hear that.

no, it's in addition to it

[aufkrawall]

Still with spawned CSMT worker in logs. :( a_file.zip

[axeldavy]

Thanks for the test.

There seems to be something fishy indeed. So far it sounds like a bug in _mesa_hash_table or the way nine uses it. I won't look into fixes right now (family holidays), but will get back to it soon.

[aufkrawall]

Enjoy your holiday, I don't mind a vacation from creating debug logs either. :)

[axeldavy]

I suspect the issue the issue must be some memory trashing due to reading freed data.

I'm refactoring the code, this should fix it.

[axeldavy]

It'd be great if you could confirm the issue is fixed now.

[aufkrawall]

Yep, it now renders without corruption and doesn't crash instantly after loading. However, it still crashes when walking around some areas of the game world (seems to happen often). It's definitely more instable than DXVK/wined3d, however the game doesn't seem to be fully stable when run in wine in general (even though it doesn't suffer address space issue as 64 bit program).

[axeldavy]

Well I guess we now have to fix that other crash.

[aufkrawall]

I'll try if I can capture a crashy scene with apitrace, if that makes sense.

[axeldavy]

Two solutions here: Either capture the logs, or capture a trace with wined3d and hoping replaying with nine will crash like when playing.

[aufkrawall]

I've captured a trace with wined3d, though replaying it with any D3D9 wrapper is even more crashy than when in-game (it didn't crash while capturing though): https://drive.google.com/file/d/1eNmNokRSymZcweo9gqZ8l8euj3ggQlaf/view?usp=sharing

[axeldavy]

Unfortunately the trace has recording issues, as apitrace has trouble with double unlock. It does memcpy outside of bounds. I guess unless they fix it (https://github.com/apitrace/apitrace/issues/697), we cannot use apitrace to debug this crash.

You'd need to produce a log with NINE_DEBUG=all csmt_force=0 Mesa needs to be built with asserts (-D b_ndebug=false) or debug on for this to do anything (the output will be big).

[aufkrawall]

Ok. Do I have to apply patches? Last time there was the issue of undesired csmt workers still spawning.