Thief 2 The Metal Age and rendering problems

[fl82]

Hello, I was trying ddraw with the Thief 2 game in Win 8.1 with also the modern patch that's needed to make it work into new o.s. using the older legacy Directx6 renderer (commenting the ";use_d3d_display" line in the cam_ext file of the patch while many other gpu would be able to use the newer DX9 renderer that unfortunately on this gpu bring many other gfx problems and in that case ddraw.dll seems to not be even called not leaving any log); before this ddraw dll, cause my gpu/drivers it wasn't able to run in real fullscreen mode (probably cause some incompatibilties with 16bit unsupported resolutions and only worked with full-windowed borderless mode with low fps). Trying this dll now it seems to start real fullscreen with 32bit colors still with legacy Directx6 engine but better frame rates but I can see strange polygons gfx errors appearing and disappearing depending on the movements and point of view. The config is an Atom dual core with a GMA3600 gpu / SGX545 core. An example of the problem that didn't appear before and now appear also in both fullscreen and windowed mode; this appears on the floor textures or walls ones becoming dark and then textures appear again seems depending on the point of view and the lights:

This is the log:

0f84 18:34:19.335 Loading DDrawCompat dynamically from C:\Games\Thief2\ddraw.dll 0f84 18:34:19.342 DDrawCompat loaded successfully 0f84 18:34:19.343 Installing display mode hooks 0f84 18:34:19.345 Installing registry hooks 0f84 18:34:19.345 Installing Direct3D driver hooks 0f84 18:34:19.346 Installing Win32 hooks 0f84 18:34:19.370 Hooking user mode display driver: igdumd32.dll 0f84 18:34:19.420 Dynamic vertex buffers are not available 0f84 18:34:19.420 Dynamic index buffers are not available 0f84 18:34:19.437 Checking source color key support: passed 0f84 18:34:19.459 Installing DirectDraw hooks 0f84 18:34:19.467 Installing Direct3D hooks 0f84 18:34:19.490 Installing GDI hooks 0f84 18:34:19.531 Warning: Failed to hook the following user32 window procedures: [buttonWndProcA,buttonWndProcW,comboBoxWndProcA,comboBoxWndProcW,editWndProcA,editWndProcW,listBoxWndProcA,listBoxWndProcW,mdiClientWndProcA,mdiClientWndProcW,staticWndProcA,staticWndProcW,comboListBoxWndProcA,comboListBoxWndProcW,scrollBarWndProcW,menuWndProcW] 0f84 18:34:19.546 Finished installing hooks 0f84 18:34:19.617 Incorrect z-buffer bit depth capabilities detected for "Intel(R) Graphics Media Accelerator 3600 Series" / Direct3D HAL; changed from "16, 24" to "16" 0f84 18:35:33.408 Detaching DDrawCompat due to process termination 0f84 18:35:33.409 DDrawCompat detached successfully

The gpu is quite complicated and might be difficult to make it perfectly working but any advice would help. :) Thanks

[narzoul]

Could you attach a save game also that reproduces the problem as soon as it is loaded? I tried running around for a while at the start of the game but didn't find any problems.

[fl82]

For example I attach this first level savegame (it's needed T2Fix community made patch for the savegame to run I suppose) moment at the very beginning once entered the first door into the castle, where the left and front wall should be lighted grey texture and not completely black or partial black triangles/areas; happens on various floors, other times seems like the light from the wall behind bleed through to opposite side of it like walls weren't "attached" themself. I only tested a few time this first level. This happens only with the DDrawCompat with the commented _camext file line as said above to run the legacy DX6 renderer of the T2Fix patch, because if it's not commented it run the DX9 modern renderer I can't use cause other still unsolved gpu related problems (unreadable directdraw menus on DX9 while game runs much faster, hangs for few seconds all the times etc..). But at least with the legacy DX6 renderer works and before without the ddraw.dll it ran without gfx errors but really really slow in windowed only mode; now thank to this dll it runs faster into fullscreen-real mode but with this unexpected problem (even if I force again the game into windowed mode the same happens). Strangely the first Thief Gold game with the TFix patch plus the same ddraw.dll, while having a similar older game engine, doesn't show any gfx problems in DX6 and it runs flawless. I imagine they used some modern lighting/texture/rendering feature for the second engine that is sensible to this complex iGPU that had difficult time rendering many games but it's curious how Thief Gold has no texture or light bleeding through walls problems with ddraw.dll while in this second game having a very similar engine this happens. I have also eventually a part of the debug dll log file of what I suppose to be the Direct3D log lines. I can only notice various lines that might say something (?) like:

[example] 13:37:38.782 Flushing 33 primitives of type 4

in the middle of the many SetTextures, SetTextureStage and DrawPrimitive or SetRenderStage lines. (but the numbers of primitive and type change even if I didn't move the player and immediately exiting the game).

[narzoul]

Please send me the debug logs also. Just load the save game you sent and exit without moving/looking around.

[fl82]

Here's the debug log compressed, just launching, loading, rendering few seconds and exiting from the game. Log Unfortunately this iGPU had a very basic control panel where just few driver options can be changed. But the fact that without the ddraw.dll in the original T2Fix DX6 windowed mode beside with a very low frame rate, the 3D texture/lighting rendering was correct in every levels, make me think it might be possible to solve it I hope.

On the other side the DX9 renderer instead I lost hope on that, there're too many problems on various side with this iGPU.. 2D menu rendered with D3D9 get mixed/unreadable and the game when run ironically it runs maybe at triple the frame rate (still a native Directx9/10 gpu) but it stops/hangs few seconds every few seconds without any logic. It'd need both an "alternative" way to draw the menu with D3D9 that replaced the DirectDraw functions I suppose and a debug study on the driver during the 3D rendering. But I'll be happy if the DX6 legacy engine might run as good as in Thief Gold. Thanks!

[narzoul]

The first frame of that save game is rendering completely differently on your system and mine based on the logs. I think we're using different patches or settings, so it's impossible to compare the logs.

What is your game version before installing T2Fix? Which version of T2Fix did you install exactly, and which components did you select during installation? Any other patches or mods?

Also, attach your cam_ext.cfg and tell me your video settings in-game, please.

[fl82]

I suppose probably because in Thief 2 I usually install the latest v118 official patch before installing the latest v.127d T2Fix. This because if I remember correctly that official v118 patch added some things into the game (like those 3D objects in front of the fire in the main hall of the mansion) and other objects that without the patch didn't appear. While I keep this installation, I'm trying reinstalling also the game from zero without the patch v118 and only the T2Fix+ddraw.dll with minimal setting (just the engine and obviously no graphic/sound modification I usually don't install anyway). The original genuine game cd that I own had the 1.07 game version in the double cd version (brown and black cd's). Meanwhile I attach the old cam_ext file setting I was using that I tried tweaking many times but in previous installation to see if I could solve this problem. For example I even tried taking the whole "texture" and "render" cam_ext part of the Thief Gold-TFix patch that runs ok on this config and used that but I suppose is similar if not identical to the T2Fix patch and in fact nothing improved.

cam_ext (new test next post)

[fl82]

As said I tried also reinstalling the game v.107 from CDs with directly only T2Fix v.127d with Basic config installation, then modifying the cam_ext commenting the d3d_display line and set 2x anisotropic filter instead of the heavy 16x one, then added your ddraw.dll from Debug-folder version and then added the savegame. Unfortunately same result maybe even darker walls textures. I attach the logs without using any other patches and without previous v.118 patch. The video game setting here is the basic autoconfigurated one (1024x768 with 32bit color of the T2Fix, no sound acceleration with only the above cam_ext mods), the video card control panel instead has almost no settings to change beside the Anisotropic filtering "app based" or "manually based" slider, texture "performance" or "quality" and the Vertex hw or sw options, but they doesn't seems to change a lot in games. These are the game 'cam.cfg' config:

_trait_cache_flags 2 character_detail 1 sfx_device 1 game_screen_depth 32 gamma 1. game_hardware 1 sfx_channels 16 game_screen_flags 45 game_full_screen 1 master_volume -1 skip_starting_checks render_weather 1 fogging 1 enhanced_sky 1 game_screen_size 1024 768 sfx_vol_3d 0 reverse_stereo 0 sfx_vol_2d 0 skip_intro sfx_vol_music 0 ambientvolume 0

Here're the logs:

Log new cam_ext new Savegame file

I also exported the Directx9 features test of the iGPU driver if it can help:

Vertex Shader Version 3.0 Pixel Shader Version 3.0

Texture Limits Max Texture Width 4096 Max Texture Height 4096 Max Volume Texture Extent 256 Max Texture Repeat 8192 Max Texture Aspect Ratio 4096 Max Anisotropy 16 Max Texture Blend Stages 8 Max Simultaneous Textures 8

Limits Max Active Lights 10 Max User Clip Planes 6 Max Vertex Blend Matrices 4 Max Point Size 511.0 Max Primitives for DrawPrimitive 1398101 Max Streams 16 Max Stream Stride 4096 Max Vertex Shader Constants 256 Max Simultaneous Render Targets 4

Caps Read Current Scanline Yes Overlay DDI No Autogen Mipmap Yes Calibrate Gamma No Share Resource Yes Manage Resource No Dynamic Textures Yes Fullscreen Gamma No Flip/Discard width Alpha Yes Copy to Vidmem Yes Copy to Sysmem Yes DXVA-HD DDI No Windowed sRGB Gamma Correction Yes Presentation Intervals Immediate, One, Two, Three, Four Hardware Cursor High-res

Device Caps Blit from Sys to Nonlocal Yes Can render after Flip Yes DirectX 5 Compliant Yes DirectX 7 Compliant Yes DrawPrimitive-aware Yes Execeute Buffers from Sysmem Yes Execeute Buffers from Vidmem Yes Hardware Rasterization Yes Hardware Transform and Lighting Yes N Patches No Hardware Rasterization, Shading, Transform and Lighting Yes Quintic Bézier Curves and B-Splines No Rectangular and Triangular Patches No Uncached Patches drawn efficiently No Separate Texture Memories No Retrieve Textures from Sysmem No Retrieve Textures from Vidmem Yes T&L Vertices from Sysmem Yes T&L Vertices from Vidmem No

Raster Caps Anisotropic Filtering Yes Interates Color Perspective correctly No Dithering No Depth Bias No Range Fog Yes Lookup Fog No Vertex Fog Yes LOD Bias No Multisample toggle No Scissor Test No Slope-Scale based Depth Bias No W-Buffer Yes W-Based Fog No Z-Bufferless HSR No Z-Based Fog No Z-Test Yes

Texture Caps Alpha in Texture Pixels Yes Draw Alpha from Texture Palette Yes Cube Textures Yes Cubemaps must be Power of Two Yes Mipmapped Cube Textures Yes Mipmapped Textures Yes Mipmapped Volume Textures Yes Conditionally not Power of Two Textures Yes Projected bump-env Lookups Yes Perspective Correction Yes Textures must be Power of Two No Projected Texture Transformations Yes Square Textures only No Tex Repeat not Scaled by Size Yes Volume Textures Yes Volume textures must be Power of Two Yes

Shader Caps Alpha for Gouraud-blending Yes Colored Gouraud-shading Yes Fog with Gouraud-shading Yes Goraud-shaded Specular Highlights Yes

Source Blending Blend Factor Yes Both Inv Src Alpha Yes Both Src Alpha Yes Dest Alpha Yes Dest Color Yes Inv Dest Alpha Yes Inv Dest Color Yes Inv Src Alpha Yes Inv Src Color Yes Inv Src Color 2 No One Yes Src Alpha Yes Src Alpha Sat Yes Src Color Yes Src Color 2 No Zero Yes

Destination Blending Blend Factor Yes Both Inv Src Alpha No Both Src Alpha No Dest Alpha Yes Dest Color Yes Inv Dest Alpha Yes Inv Dest Color Yes Inv Src Alpha Yes Inv Src Color Yes Inv Src Color 2 No One Yes Src Alpha Yes Src Alpha Sat Yes Src Color Yes Src Color 2 No Zero Yes

Z-Buffer Comparison Always Yes Equal Yes Greater Yes Greater Equal Yes Less Yes Less Equal Yes Never Yes Not Equal Yes

Alpha-Test Comparison Always Yes Equal Yes Greater Yes Greater Equal Yes Less Yes Less Equal Yes Never Yes Not Equal Yes

Thanks.

[narzoul]

Thanks! After some experimenting, I think the issue is related to Z-buffering. The game selects the D24X8 depth buffer format for you. If I force mine to use the same, I get similar display glitches as you on my AMD GPU, but in different places. On Intel HD 4600, there are only smaller glitches, like random lines of various thickness missing from polygons.

There is a "wr_render_zcomp" option in cam_ext.cfg. Commenting it seems to help on my end, does it fix the issue for you?

I'll do some further checks to see if this depth buffer format is broken because of DDrawCompat, or it's just driver issues.

[fl82]

I tried commenting the wr_render_zcomp line but there's the same problem. I attach the related debug log. The strange thing is that beside the slow almost unplayable fps with windowed mode without ddraw.dll, the texturing was correct or at least I didn't see visible major problems on all the level I finished the game time ago (maybe only the missing fog/weather effects I suppose for the lacks of some features on driver level). So maybe there's something that can solve it. :)

no_wr_zbuffer log

[narzoul]

Ok, it looks like commenting that line doesn't completely disable z-buffering either. Let's try a different approach then. This modified experimental release disables all depth buffer formats, except D16: ddraw.zip (diff.txt compared to 4370990)

I'll need the debug logs only if this doesn't fix the issue either. The wr_render_zcomp option can be enabled for this.

[fl82]

Thanks but still it shows the same problem. Here's the log. No Zbuffer Log

[fl82]

If this can help, I tried loading the latest (the previous with zbuffer enabled) ddraw into the Thief 1 Demo game with similar config, TFix+ddraw-debug where it works perfectly without gfx errors, maybe it says what miss from the different texturing problems. Thief-1-Demo Debug Log

[narzoul]

The reason I suspected the z-buffer is because on the last screenshot above, the "hole" on the wall in the back is not aligned with the wireframe of the scene, so it looks as if only parts of polygons are missing/corrupt. Here's what the wireframe should look like after loading your last save game, without moving the camera:

Here's the patch I used for rendering it: ddraw.zip (diff.txt on top of 4370990)

What does it look like on your end? Are the wireframe edges aligned with the glitches?

[narzoul]

By the way, does DDrawCompat v.0.2.1 not work at all? Does it have the same glitches?

[fl82]

It's strange with this wireframe version it seems that some polygons are "covered" but can be seen in the angles (?) in my rendering see the attached screenshot.. also with latest v1.07 new game installation from zero, those black shadow are even bigger and vertical on both walls and not always following the polygons lines or at least not all the times.

Log wireframe

UPDATE: I can't believe this but with the older official version ddraw v0.2.1 it works and the rendering of the scene is complete! I was sure I tested that before but maybe some previous installation had others problems; I hope anyway this can help to understand what in this newer version would not work on this iGPU.

UPDATE2: When I was thinking it was solved something other strange things happens: continuing the game all seems to be ok. The first torch with the water arrow closed the hall light and walked up to the main salon of the mansion where there were the fire, four sofas and some vases.. all was ok. Then I tried to improve a bit the frame rate, changing the multisampling into the cam_ext that anyway should not change anything in the DX6 renderer but I saw it was passed into the cam.cfg variables. Incredibly now the torch of the hall with the two guards when used the water arrow shut down but the hall remains lighted with the torch off (and the guards see the player) and up in the the salon the sofas and vases are disappered! I tried using again the multisampling 8 value again but that remain broken.. incredible.. I suspect that just modifying with notepad (after the first run) the file somehow "break" something and that would explain why the first time I tried the older 0.2.1 ddraw.dll had similar problems cause I'm sure before modifying it both the torch lightmapped area went off and the sofas and vases were there! How can this be possible I'm thinking.. The only thing is trying again to reinstall again from zero and just commenting the d3d_display line.

[fl82]

UPDATE3: I tried reinstalling the game and as imagined another clean installation from CDs v.107 game + T2Fix 1.27d Basic + ddraw.dll 0.2.1 + before starting the game just commenting the d3d_display line with notepad and then run the game solved again these "new" problems.

I tried exiting the game and reloading a random savegame of the new installation and these until now things are still ok. So I suspect that cam_ext.cfg file with my config or who knows why, maybe should be modified with some specific editor or maybe only before the first run of the game cause after that it might conflict with something. Interesting discovery anyway and I'll do some test again with the newer ddraw.dll at this point. The frame rate is really higher than the windowed mode but still not smooth in the wide geometric areas. The DX9 renderer would solve that but as said, I suspect gpu drivers doesn't like both the new way D3D9 render the 2D menu and videos and once the 3D rendering start it runs smooth for a while than stop then again it unlock itself and then stop for few seconds and so on... a pain to debug I suppose.

[narzoul]

It looks like I was on the wrong track with the z-buffer. As it turns out, my GPUs don't actually support the D24X8 format, but the drivers don't fail the resource creation with it either. So it's not that surprising that the end result was broken.

It's probably one of the 3D optimizations I added to the WDDM driver hooks that broke it after v.0.2.1 for you. If you're still willing to help with debugging this, please try this version next: ddraw.zip (diff.txt on top of 4370990) This will disable the suppression of redundant rendering state changes.

[fl82]

I tried this version on the clean installation I tried above the v0.2.1 not modifying again the cam_ext, and as imagined this works, no gfx errors, no lightmap area problems and sofas and vases remains in the fireplace of the mansion salon. :) But I suspect the strange problem about what happens if I modifying after the first run the cam_ext had something to do in previous tests. I'm trying to understand this. Anyway it works on a clean installation! :)

[narzoul]

So does the latest "official" experimental release also work on a clean install?

[fl82]

The latest ddraw attached above, yes I've just launched with it and it works. Arrived until the mansion salon and didn't see any gfx errors. But later I'll do more test and write if I see anything wrong. :)

[narzoul]

No, not the ones attached in these comments. I mean the one here: https://github.com/narzoul/DDrawCompat/releases/experimental

[fl82]

Ah sorry, I'll try later and update here.

[fl82]

Here I am. I tried using the experimental version linked (the 18/12/2020 dll) and no it does not work like before even in the clean installation directory.

I will try using again the above dll's and see if it come back the correct rendering.

[fl82]

I tried also in the same directory (not reinstalling the game but not modifying any settings or cam_ext) using again the official v0.2.1 old release and it renders ok back again; so I suppose only it does not with the linked experimental version it seems (beside the cam_ext problem that is probably related to something else).

Old official release v0.2.1 (not the experimental):

[narzoul]

Ok, so since the last attached release (in the comments) was working, it definitely has something to do with one of the render state changing functions, just have to figure out which one. From the logs, my first guess is pfnSetVertexShaderDecl, which is always called by the runtime after pfnSetStreamSource. If your driver resets the vertex shader declaration after changing stream sources, that would explain the malformed polygons.

Here's a version with pfnSetVertexShaderDecl removed from state caching: ddraw.zip (diff.txt on top of 4370990)

[fl82]

Tried the above attached dll with no pfnSetVertexShader and like the experimental same rendering result. I notice some differences from the experimental rendering like the missing shadow under the table.

[narzoul]

Sorry, this is going to be a bit of a grind. I compiled 10 different versions here, to test each of the potential state changing functions that could break things. The diffs are in the attachment, but basically all I did in each case was that I replaced the other 9 functions' macros with FLUSH_PRIMITIVES instead of SET_DEVICE_STATE_FUNC, so that only that one remaining function will be cached.

Please let me know which of them have any glitches. All of them are in this zip: ddraw.zip (all diffs are on top of 4370990)

[fl82]

I tried them all and only one have the rendering problems and I attach the screenshot of the first mansion room with that single dll into the pfnSetRenderState folder.

The others dll rendering are all ok on a fast test.. no gfx errors seen in any of them at least for this fast test.

[fl82]

Meanwhile I'd like to ask you a curiosity that is not related to ddraw.dll and about what happens with this game using the d3d_display line and basically the D3D9 renderer as you can see in the patch, the D3DX9_43.dll file I suppose. My iGPU (GMA3600/SGX545 core) was a native Directx9.0c/10 low power gpu with 400Mhz core and many "high end" features for netbook of those times but at the end had problems with speed, drivers and compatibility. Anyway should make this game to fly considering how old it is but both the o.s. and the gpu itself seems to not be oriented for old games considering Directx9 complex games runs just as fast as old Directx6 if not faster. If I enable the D3D9 renderer in both videos and menu the following tiled effect happens changing colors when moving mouse or randomly but making impossible to read or select anything originally 2D, if not randomly:

IF I get to start the 3D rendering game, at first it starts ok at really really fast frame rates with no gfx error but after few seconds it hangs for few seconds than unlock itself depending on the movement and the frontal scene complexity (for example if the player looks to the floor only it seems easier to wake up and come back to the fast rendering, if the scene is complex it often hangs for seconds and unlock and then pause again and then unlock an so on. The only gfx errors would be the menu and video itself while the 3D rendering problem seems related to some other problem but the rendering itself would be perfect and like double/triple the speed. Starting from the menu... I understand that Directx9 draw that DirectDraw menu using some 3D function that "compensate/replace" it and probably my gpu drivers doesn't support or not perfectly. How could I solve that for start if you know about Directx9 differences? Obviously ddraw.dll I imagine can't work with Directx9 but I am curious to understand what could be the reason of that confused menu that doesn't happen with any other gpu I tried. Even an old an slower Geforce FX 5200 PCI works perfectly with the D3D9 renderer of this patched game. I suppose it'd be needed some alternative way to render the menu and videos but the engine I suppose will not be updated and surely not the gpu drivers (last are of the 2013).

[narzoul]

It's a little strange that pfnSetRenderState is causing the issues. During the 3D part of the game, that function isn't actually affected by the caching at all. So maybe some render state breaks during the transition to 3D and remains stuck in the wrong state.

I noticed from the logs that the game creates 2 Direct3D devices. The second one is created during the transition, so maybe some state is not reset properly there. Here's a possible fix that clears the full state cache whenever the render target changes: ddraw.zip (diff.txt on top of 4370990)

Direct3D 8 and newer are completely separate interfaces, independent of DirectDraw, which is why they're not supported by DDrawCompat. Most of the 2D stuff that was earlier provided by DirectDraw was removed from them, and some of it was moved to new APIs. As far as I know, the proper way to render 2D in Direct3D 8/9 is just to render textured rectangles in 3D (think of it like rendering a wall close the camera).

It is best to ask the developers of the engine for T2Fix to look at those menu issues and freezes. They have access to the source code and know how it works. For everyone else it's a much more difficult task.

If you still want to give it a shot, I'd recommend trying out apitrace: https://apitrace.github.io/ It's a very powerful tool for capturing traces of many different graphic APIs, and also supports Direct3D 9. There is plenty of info on the home page, but basically you can download the 32 bit Windows binaries, copy the lib\wrappers\d3d9.dll to your Thief 2 install directory and run the game. It'll collect the trace to Thief2.trace on your desktop, which you can then open with bin\qapitrace.exe. It can even replay the trace and display what those API calls should render on your screen. Apparently you can even edit the calls.

[fl82]

I tried the attached dll above and still the same problem of the pfnSetRenderState version:

Regarding the Directx9, thanks for the hints I'll try to understand those problems even if I've limits understanding things at this code level but still it's an opportunity to know more about how things works. I could use older more compatible system or modern faster one but I've got this mini-itx board into a mini-itx case and it's still an interesting x64/x86 config that I always thought was not pushed as fast as it could and most of all the GPU. It had a long story of strange incompatibilities and problems that was never clear why considering the on paper specifications. At the end it was discontinued and replaced by other modern Intel Graphics cores and this was left with only a theorical Win 7 32bit compatibility. For example with your ddraw.dll the 3DMark2000 benchmark of its time doubled the previous result score on Win 8.1, just to imagine there was quite a margin of optimization.

[narzoul]

Ok, let's take a step back and double check that really only pfnSetRenderState causes problems. This patch removes only this function from caching: ddraw.zip (diff.txt on top of 4370990)

If there are no glitches with this, then we're gonna have to narrow down which particular render states cause the problem. There are dozens of them...

[narzoul]

I may have accidentally attached the wrong build in the previous comment. I updated it now, just in case. Please redownload it if you already got it.

[fl82]

I tried the latest attached above dll and it works and no gfx errors.

Also I tried as suggested the apitrace way for the D3D9 rendering. It makes a big trace file even with only a few seconds of the menu gfx problem and using the qtapitrace GUI really seems like a powerful tool; the only problem is that it sure requires more knowledges and it doesn't report any major error (beside two dll loaded in the first frame with a smile) but when replaying the traced rendering with that tool, it clearly shows the usual gfx error like in real time. But indeed it's not an easy tool to use but sure impressive and I understand that from the first frame (!) rendered in the trace, the frame buffer is already full of random pixels and rectangles. I was thinking to compare line by line the trace file but I don't have a functional traced similar file done in the menu only (entering and exiting) and anyway I'm not sure I'd understand it. But I think the problem is in the first two frame of the traced calls, at least for the menu problem.

[narzoul]

Here's the next puzzle: ddraw.zip (diff.txt on top of 4370990)

The zip now contains a renderstates.txt file too. Please copy this also to the Thief 2 install directory.

It has a bunch of numbers in it. Each one of them is an index of one of the render states that pfnSetRenderState can modify. The ones that are listed in the file are not going to be cached by DDrawCompat. By default, it contains all render states, so running the game with it should produce no graphics errors (please verify that). Removing all numbers should reproduce the usual glitches.

The goal is to reduce the file contents to the minimum set of render states that need to be kept there to avoid glitches. Please note that the file can only contain numbers and whitespaces (empty lines are ok).

Since there are 107 entries, testing them one by one would take a while. I'd recommend trying a faster strategy. For example, move half of the entries to a second file and run a test with the remaining entries. If there are no graphics errors, then you can delete the contents of the second file and continue by moving half of the remaining entries to the second file again. If there are graphics errors, then you moved one or more of the needed render states, so you should try to figure which ones are those by moving again half back to renderstates.txt, etc. On each successful test, everything in the second file can be deleted.

[narzoul]

I've played around a bit with apitrace in the meantime, so here are some further tips.

If you double click a function call in the trace, a pane will open on the right side with some info about the state of the rendering just after that call. (You can also double click the frame header, which I assume shows the state after the last call in that frame). On the bottom of the pane, switch to the "Current State" tab, then select the "Surfaces" tab at the top. Here you will see thumbnails of the currently used textures and framebuffers. You can double click the thumbnails to show the full sized image.

Generally, you should expect the RENDER_TARGET_0 framebuffer to contain what will be displayed on your screen, and most probably it will only be updated by the IDirect3DDevice9::DrawPrimitiveUP calls (towards the bottom of each frame except the first), so try double clicking those calls. The PS_RESOURCE_0_LEVEL_0 texture should show a 640x480 image of the main menu. The RENDER_TARGET_0 framebuffer should simply be a scaled up version of the same (1024x768). I assume this is what gets corrupted in your case, while the texture should contain the correct image (I think the texture is generated in software, without using the GPU).

For me, frame 1 has solid white texture with solid black framebuffer. Frame 2 has the background of the main menu without any text in both. Frame 3 also has the text and the mouse cursor in both. The rest continues in the same way, tracking the mouse cursor movement.

[fl82]

Ok, I'll update as soon as finished the test.

For the D3D9 trace, in my early frames there's a complete no-sense of pixels/rectangles and portion of the main menu colors or the Win GUI (for example the numbers of the right angle date/time repeated in many tiles after the Frame 1. The Frame 0 is always completely black. But the first strange thing I notice is that every time I double click on the initial frames for example, each double click from the frame 1 to the frame 5 (example) result in a different rendered image that is, as said sometime lines or part of the "2021" time/date/font/background windows GUI color of the right angle, multiplied in that single frame and that will maybe change colors or pixels next time double click on it (example of one of the early frames):

and one the first frame that will be different probably next time I'll double click on the frame 1.

When running in real time it's probably so fast that most of the rendered images are mixed rectangles/tiles that sort of remember the main menu brown colors... but it's completely random in single frame mode I suppose.

[narzoul]

What if you double click on the IDirect3DDevice9::SetTexture calls? Is the PS_RESOURCE_0_LEVEL_0 texture corrupted in those too, and randomly changing even in the same frame?

[fl82]

Interesting, in the frame 1 the SetTexture call open as PS_RESOURCE a clean grey and white square image (but I suppose is trasparent or white or empty) and the frame buffer is black. In the frame 2 the SetTexture call open as frame buffer a confused rendering as usual, but the PS_RESOURCE image is definetely the good game main menu background image as supposed to be.

[narzoul]

I didn't think it was possible, but apparently you can play back the trace even on a different GPU. I just tried playing back a trace on my Intel HD GPU that was captured on my AMD GPU, and the frames are produced correctly. Can you send me your trace file? I can take a quick look at it.

[fl82]

Each time I double click on SetTexture the frame buffer still change every time.. Here's the trace: T2 Menu trace

[narzoul]

Interesting, your trace renders correctly on my end. I think it's DrawPrimitiveUP call which somehow does the rendering incorrectly for you, maybe it uses the wrong texture (or wrong part of the "video memory") as input.

What happens if you double click on the DrawPrimitiveUP calls in frame 1 and frame 2? The texture should be empty (transparent) in frame 1 and should show the menu background in frame 2. I assume RENDER_TARGET_0 shows garbage in both calls for you, correct?

Try editing both of those calls (right click -> Edit), and change the PrimitiveCount to 1 in each. Then double click each of them again, does it change the results in RENDER_TARGET_0? It should only render into the top right triangle now. If the bottom left triangle remains black, then it's only the texturing/rasterizing part that's buggy. I wonder why it works correctly in the 3D parts of the game.

[fl82]

I edited both PrimitiveCount from 2 before to 1 now and double clicked again, shows still some random confusing pixel on the left high angle and sometimes on the right area and right low angle but most of the rendered image is now black with these triangles in RENDER_TARGET_0.

Meanwhile on the other side I did some initial test for the legacy renderer with the above DDrawCompat and already discovered that: 1) with the complete txt file with all the number it works ok 2) it can work even with only the 129 value in the renderstates.txt. It's still early to say but it looks like one if not the only one that works but I'll do more test. Most seems to break as usual. But might the ddraw.dll need in other moments some of the others values? I can't see any gfx errors with just the 129 value but I wonder if the others values might be needed in other moments?

[narzoul]

129 makes no sense to me. It's D3DDDIRS_WRAP1. There are also D3DDDIRS_WRAP0 through D3DDDIRS_WRAP7 that could be related then, but it's enough if you check the two neighbouring ones for now I think (128 and 130), with just that one number in the file in each case.

[fl82]

I still didn't try combination of those values but once divided the list in different files I had to test them number by number to understand if / which one worked alone and for luck I found sooner than expected that 129 worked alone in the txt file.

With only 128 into renderstates.txt usual problems:

With only 130 value same result:

With only 129 value:

I can't explain how and why it happens but I'll continue trying the other values. :)

[fl82]

I am trying a lot of single value numbers into the txt file and until now I didn't find any other value that can work alone as unique number in the txt for half the list. I tested also If I understood correctly that saving only both 128+129 works ok. Saving only both 129+130 works ok too. But of these three numbers (128,129,130) only the 129 seems could render ok if saved without other numbers. But I'll continue to run with the other values if there're others that can work as single number in the renderstates.txt.

[narzoul]

Ok, after some testing with the WRAPn render states, I have a theory of what the problem is. In your log (and also mine), there are some strange values assigned to WRAP0:

0cb0 11:19:52.294       > _D3DDDI_DEVICEFUNCS::pfnSetRenderState(00D06F54, {D3DDDIRS_WRAP0,4294967294})
0cb0 11:19:52.294       < _D3DDDI_DEVICEFUNCS::pfnSetRenderState(00D06F54, {D3DDDIRS_WRAP0,4294967294}) = 0
0cb0 11:19:52.294       > _D3DDDI_DEVICEFUNCS::pfnSetRenderState(00D06F54, {D3DDDIRS_WRAP0,0})
0cb0 11:19:52.294       < _D3DDDI_DEVICEFUNCS::pfnSetRenderState(00D06F54, {D3DDDIRS_WRAP0,0}) = 0
0cb0 11:19:52.294       > _D3DDDI_DEVICEFUNCS::pfnSetRenderState(00D06F54, {D3DDDIRS_WRAP0,4294967292})
0cb0 11:19:52.294       < _D3DDDI_DEVICEFUNCS::pfnSetRenderState(00D06F54, {D3DDDIRS_WRAP0,4294967292}) = 0
0cb0 11:19:52.295       > _D3DDDI_DEVICEFUNCS::pfnSetRenderState(00D06F54, {D3DDDIRS_WRAP0,0})
0cb0 11:19:52.295       < _D3DDDI_DEVICEFUNCS::pfnSetRenderState(00D06F54, {D3DDDIRS_WRAP0,0}) = 0

The above values are 0xFFFFFFFE and 0xFFFFFFFC, i.e. all but the last 1 and 2 bits are set, respectively. According to the docs, only the 4 lowest bits have any meaning. It doesn't even make any sense, since the state is immediately reset to 0 afterwards, but I see the same behavior in other games too. These values are not set by the games, they seem to be coming internally from the Direct3D runtime.

I guess on your driver, the other bits (accidentally?) get assigned to the WRAP1 - WRAP7 render states, maybe because the driver stores them all in a single 32-bit value and some bitwise operation overwrites the higher bits too. But on both my AMD and Intel drivers, all but the 4 lowest bits are ignored.

So when I added the render state caching in DDrawCompat, these calls on your driver "corrupted" the WRAP1 - WRAP7 states too, which the cache didn't know about. When those states were reset to 0, the cache thought they were already at 0 and ignored those changes. The WRAP2 - WRAP7 states are not used by this game, so they don't cause further problems, but the WRAP1 corruption causes those strange shadows. I can now reproduce similar glitches when emulating this "overflow" in my drivers. Although in my case the glitches are along the polygon edges, and on your system it's not always the case, which I still don't understand.

I've added a small fix on top of the latest experimental version, which discards all but the 4 lowest bits from these states, so hopefully the behavior will be identical on all drivers now: ddraw.zip (diff.txt on top of 4370990)

[fl82]

Very interesting! This gpu and driver always has been complicated for gaming (even retrogaming imho but it's also cause modern o.s./api seems to not be really oriented for retrogaming) and mostly for netbooks light usage; I always imagined that its core has been originally oriented to smartphone (many Cortex A9 ARM SoC had the previous SGX535 core). The drivers were done only for Win 7 32bit even if the cpu was already x64 and sort of works into Win 8.x 32bit system (they should be compatible to WDDM 1.1). It saw only few years of driver updates not to mention the linux story.

Anyway, to complete the previous dll test (even if I might not be sure 100%) all the other numbers tested if saved alone without any others in the renderingstate.txt, running the game showed the gfx artifacts. If used the original full list in renderingstates.txt and removed only the 129 value equally it shows the artifacts. If removed from the full list each 128 and 130 but leaving inside the 129 value, it works both times and as seen even with only that. :)

Now I try the latest dll above. :-)

EDIT: the latest attached in the above comment works.

[fl82]

Meanwhile to continue understanding also the D3D9 renderer problem I'll try to trace the part of the 3D rendering initial game even if it's difficult to arrive there with random mouse clicks for the menu problem. :-)

EDIT: I launched the game quite fast with few seconds of the traced D3D9 3D rendering. I captured with ffmpeg a video of few seconds of the game rendering with D3D9 engine from the confused menu to the 3D rendering and back to the menu and it can be seen also how it already pause/hangs for few seconds too.

[narzoul]

Added the fix to the latest experimental release. Thanks for the extensive testing!