Thief 2: Configuration issue - how do I set a value?

BEENNath58 commented 4 weeks ago

Suppose I wish to use Frame Limiter, how do I specify a value for FpsLimiter The line is FpsLimiter=flipend

Where and how do I put the MaxFPS value?

The game is unstable at over 60FPS. I need this to test if DDrawCompat can fix the Gamma controls for Thief 2. The gamma controls don't work, even with the AllowMaximizedWindowedGamma shim

narzoul commented 4 weeks ago

In parentheses, like flipend(60). But the default is 60 anyway, so it's fine as you put it too. You can change it also from the config overlay (Shift+F11).

BEENNath58 commented 3 weeks ago

It looks like the problem was the debug dll, that freezes the game or slows it down. We are talking about Thief 2 vanilla

I am surprised to see the in-game stars are back in 0,5.0. How did you manage it? 0.4.0 didn't work with the game, 0.3.2 had no stars.

There is a whole discussion behind the actual issue: https://www.vogons.org/viewtopic.php?p=996195#p996195

narzoul commented 3 weeks ago

It looks like the problem was the debug dll, that freezes the game or slows it down.

The debug build has LogLevel=debug set by default, but even without it it would be slower than a release build due to additional debug checks inserted by the compiler.

I am surprised to see the in-game stars are back in 0,5.0. How did you manage it?

I don't remember doing anything for that particular issue. I remember reading theories about the game using palettized textures and that causing the missing stars, but I never found any evidence of that. There was no sign of any palettized texture usage, even when support for it was enabled in DDrawCompat.

I have the GOG version of the game, not sure what patches are installed on it by default. I'll see if the issue can be reproduced with it.

narzoul commented 3 weeks ago

It seems the issue was fixed by https://github.com/narzoul/DDrawCompat/commit/8b393ea49876f8f7ee85182304bc265583b816c4. More specifically, removing the o.pos.z = saturate(o.pos.z); line from VertexFixup.hlsl makes the stars disappear again.

The saturate function clamps its argument between 0 and 1, so the game must be setting the Z coordinates of stars outside of that range. Some quick debugging on top of v0.3.2 reveals that those coordinates are in the range of thousands. Whether the driver is actually supposed to clip such vertices or not, I'm not sure.

Here's the test build I was using, which isn't using the VertexFixup.hlsl yet, instead it fixes vertex coordinates via the CPU: ddraw.zip (diff.txt compared to v0.3.2) I added a quick fix into it, which clamps Z > 1 to 1, and also logs the coordinates of all problematic vertices in the log file. So don't look at the stars for too long, or the log file will be very big.

BEENNath58 commented 3 weeks ago

I added a quick fix into it, which clamps Z > 1 to 1, and also logs the coordinates of all problematic vertices in the log file. So don't look at the stars for too long, or the log file will be very big.

Yea its EXTREMELY big integers. Log attached DDrawCompat-thief2.zip

I have the GOG version of the game, not sure what patches are installed on it by default. I'll see if the issue can be reproduced with it.

It installs at least T2Fix, although to my knowledge there is a Direct3D6 renderer still available inside.

Another query: how did you fix the gamma? Even T2Fix (the GOG patch) couldn't do it. 0.3.2 works, 0.2.1 can't run the game.

narzoul commented 3 weeks ago

I don't think there is any fix for gamma in v0.3.2. Maybe exclusive fullscreen mode fixes it.

In later versions, when FullscreenMode is set to borderless, gamma ramps are emulated with shaders.

BEENNath58 commented 3 weeks ago

I don't think there is any fix for gamma in v0.3.2. Maybe exclusive fullscreen mode fixes it.

You are right, its the exclusive fullscreen. Thanks

As a token of thanks, here's something that's problematic in DDC but could probably be fixed. There is a game, Grand Prix 3. In HW mode, the game has pink tyres. It is seemingly caused due to the usage of textures of size which are not a power of 2

ghotik commented 3 weeks ago

Hi, thanks for the extremely interesting news. I'd like to replicate your stars fix in DxWnd. May I ask you what D3D operation is actually responsible for the problem? I tried to trim all z coordinates to the 1.0 maximum value for all vertex in the IDirect3DDevice3::DrawPrimitive method, but with no success, the stars are still missing. Maybe I should hook some other method?

narzoul commented 3 weeks ago

I'm not sure, because I'm hooking at the driver level, which handles all of them. Back when I tried to hook the various DrawPrimitive methods in the runtime, I found it to be too difficult and unpredictable, because the runtime uses some dynamic vtable and sometimes replaces some of these methods (and maybe others) with different method pointers, so it's hard to hook all variants. I gave up and settled on driver hooking instead.

Also, are you calculating the stride between vertices correctly? It depends on the FVF flags passed to DrawPrimitive. I guess if the first vertex has Z > 1, then you know you are in the right place, but the stride may still be incorrect for finding the rest of the vertices in the array.

ghotik commented 3 weeks ago

Well, that fixing at the application level is not possible is arguable, but I don't want to miss a precious chance to learn how to put my hands on the 3D vertices at the driver level, a solution, I agree with you, enormously more flexible and valid for any DirectX 3D access, so I'm going to roll up my sleeves and try it. So far I reached and hooked the pfnDrawPrimitive driver callback with a tracing proxy and it works, but the pfnDrawPrimitive seems to get some information about the drawing activity (like the number and type of elements) but it doesn't have access to the vertices data. I was trying to get inspired (a.k.a copying) by your DDrawCompat code, but in this part the architecture is so different that the operation is not easy. Could you help me clarifying some issues, that is the following:

what type of elements have to be fixed? (i.e. D3DPT_TRIANGLELIST, D3DPT_TRIANGLESTRIP, or D3DPT_TRIANGLEFAN or all three of them)
which data has to be trimmed? (the z coordinates)
what other driver callback should I hook to reach th data to be trimmed?

If you follow the DxWnd thread on SourceForge about this issue you'll see that one difficulty comes from the fact that you can't be sure about the effects of the code changes because the problem sometimes seems to vanish for completely different reasons.

narzoul commented 3 weeks ago

what type of elements have to be fixed? (i.e. D3DPT_TRIANGLELIST, D3DPT_TRIANGLESTRIP, or D3DPT_TRIANGLEFAN or all three of them)

Probably all of them, and not even just triangles.

which data has to be trimmed? (the z coordinates)

Yes, Z > 1 -> Z = 1. DDrawCompat also does Z < 0 -> Z = 0, but I don't remember if any games need that.

Then there are also various RHW fixes which you may already be aware of, but it's an unrelated issue. Some drivers don't render RHW=0 and RHW=infinite, so I replace them with RHW=1.

what other driver callback should I hook to reach th data to be trimmed?

This parts gets a bit complicated because of different vertex sources and FVF, so here's the breakdown.

Driver functions referenced below are documented at: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/d3dumddi/ns-d3dumddi-_d3dddi_devicefuncs

Vertex sources

The driver supports both straight pointers to user-mode memory (e.g. IDirect3DDevice7::DrawPrimitive) and vertex buffers (e.g. IDirect3DDevice7::DrawPrimitiveVB).

User-mode pointers are received by pfnSetStreamSourceUm in the pUMBuffer parameter. This is the same pointer which is passed in lpvVertices of IDirect3DDevice7::DrawPrimitive. Additionally, pData->Stride is the size of a vertex in the memory (determined based on the FVF code passed to DrawPrimitve, so you don't have to calculate it yourself). You can ignore pData->Stream, because for transformed vertices it should always be 0.

For vertex buffers (not used by Thief 2), the stream source is set by pfnSetStreamSource. pData->Stream and pData->Stride can be treated as above. pData->hVertexBuffer is a handle to the vertex buffer resource, created by either pfnCreateResource or pfnCreateResource2. pData->Offset can be ignored, it should always be 0 for DX7.

DDrawCompat only ever supported CPU editing of system memory vertex buffer data, so that's the case I'll focus on. To get a pointer to the vertex data for a system memory vertex buffer, you need to track its creation via pfnCreateResource(2), and free the associated tracking data when the buffer is destroyed via pfnDestroyResource. For pfnCreateResource(2), pData->Format = D3DDDIFMT_VERTEXDATA and pData->Flags.VertexBuffer = 1 for vertex buffers, and pData->Pool = D3DDDIPOOL_SYSTEMMEM to indicate it's in system memory. In this case, pData->pSurfList[0].pSysMem points to the vertex data. You'll need to save that pointer and associate it with the handle returned in pData->hResource, which matches the hVertexBuffer handle passed to pfnSetStreamSource.

The current vertex source can alternate between user-mode memory and vertex buffers, depending on which driver function was called last with valid arguments, so you need to track this information.

Flexible Vertex Format

I assume you are already familiar with FVF codes. We need this to determine if the vertex source contains transformed or untransformed vertices, because we only want to edit transformed vertices.

The driver doesn't receive FVF codes directly, but a different representation of it, which was introduced by DX9 and described in detail here: https://learn.microsoft.com/en-us/windows/win32/direct3d9/vertex-declaration

Anyway, we don't need a complicated analysis here, but some data needs to be tracked again. The vertex declaration is created by pfnCreateVertexShaderDecl. pData->NumVertexElements contains the number of elements in the pVertexElements array. The vertex declaration uses transformed vertices if any of the elements in pVertexElements contains D3DDECLUSAGE_POSITIONT in the Usage field. Save this information and associate it with the handle returned in pData->ShaderHandle. It can be released in pfnDeleteVertexShaderDecl.

The currently active vertex declaration is selected by pfnSetVertexShaderDecl. We only want to edit vertex data if the currently selected handle (hShaderHandle) is associated with transformed vertices.

Drawing non-indexed primitives

You already found that pfnDrawPrimitive handles non-indexed primitive drawing. There is also pfnDrawPrimitive2, but I don't think DX7 uses it, so DDrawCompat doesn't even hook it.

Now that you have the stream source and stride, the rest should be fairly straightforward in this case. pData->VStart is the index of the first rendered vertex, so multiply it by stride to get a byte offset into the stream source. The number of vertices rendered from this offset is not received directly, you have to calculate it from pData->PrimitiveType and pData->PrimitiveCount. See getVertexCount in D3dDdi\DrawPrimitive.cpp.

Drawing indexed primitives

This is a fair bit more complicated. The main function is pfnDrawIndexedPrimitive2. There is also pfnDrawIndexedPrimitive, but it's not used by DX7.

dwIndicesSize is always 2 for DX7, so pIndexBuffer is practically always an UINT16 array that contains the vertex indexes. The number of elements in the index buffer matches the number of vertices to be rendered, which can be calculated the same way as in the non-indexed case.

pData->BaseVertexOffset is similar to VStart in the non-indexed case, but it's expressed in bytes, so divide it by stride (it should always be divisible without remainder) to get the base vertex index. Beware that this can be negative. Then, to get the actual index of a vertex, add the index from the index buffer to the base vertex index. Multiply it by stride to get a byte offset from the start of the vertex source.

The pData->MinIndex and pData->NumVertices are supposed to indicate the range of indexes in the index buffer (MaxIndex would be MinIndex + NumVertices - 1), before applying the base vertex index. This latter part is not obvious from the driver docs, but it's explained better in the DX9 SDK: https://learn.microsoft.com/en-us/windows/win32/api/d3d9/nf-d3d9-idirect3ddevice9-drawindexedprimitive However, based on my old results, DX7 doesn't fill these fields properly for some reason, so you'll have to calculate them yourself by finding the smallest and largest index in the index buffer. See the DrawPrimitive::drawIndexed method. I found that actually populating these fields properly before calling pfnDrawIndexedPrimitive2 can increase performance on old Intel drivers, which otherwise probably ended up processing too many vertices because of the incorrect initial values in these fields.

Then, you have two options:

Edit all vertices given by MinIndex and NumVertices, which could also edit some vertices that are not actually rendered.
Edit only the vertices referenced by the index buffer, but then you could end up editing the same vertex multiple times, unless you keep track of which ones you already edited (which might not be worth it for very simple edits).

ghotik commented 3 weeks ago

Hi again and thanks so much for the very detailed and complete analysis of this complex situation. I started to gradually hook and implement some of your directions hoping in some luck, but it seems I had not enough. In any case, what I did so far is enough to point to some doubts that may need clarification. My test case is the very beginning of the Thief II starting mission where I log the operations and try to make some coarse change to the vertices that should be evident at a first glance. Here are some steps:

1) I started hooking pfnCreateResource adding some simple logs, but I got none, I supposed that this callback is not used here. 2) So I hooked pfnCreateResource2 where I got more logged operations. Here I didn't know how to interpret your sentence that follows:

For pfnCreateResource(2), pData->Format = D3DDDIFMT_VERTEXDATA and pData->Flags.VertexBuffer = 1 for vertex buffers, and pData->Pool = D3DDDIPOOL_SYSTEMMEM to indicate it's in system memory

If I have to map the word "and" in a logical and condition I get no calls that match the end of all three conditions. So I put the conditions in logical or and traced which element triggered the condition. Here is my log:

dxwnd.CreateResource2: hDevice=0xa410c0 pResource=0xdedf8
dxwnd.CreateResource2: SurfCount=1 handle=0xdbd6728
> reason: VertexBuffer
dxwnd.CreateResource2: hDevice=0xa410c0 pResource=0xdee78
dxwnd.CreateResource2: SurfCount=1 handle=0xdbd6b68
> reason: VertexBuffer
dxwnd.CreateResource2: hDevice=0xa410c0 pResource=0xdee78
dxwnd.CreateResource2: SurfCount=1 handle=0xb80db78
> reason: VertexBuffer

This should mean that the game creates three vertex buffers but none of them is created with pData->Flags.VertexBuffer set to 1 or in system memory. So, are these three vertex buffers suitable for the vertices transformation? Maybe this is a dead end for this situation and I should look for the other cases? I don't know, but I'll start to explore also the other cases.

narzoul commented 3 weeks ago

I started hooking pfnCreateResource adding some simple logs, but I got none, I supposed that this callback is not used here.

It can be used by older systems, so you should hook both whenever possible. The docs say pfnCreateResource2 is "Supported starting with Windows 8", but it might also depend on the WDDM version supported by the driver.

More precisely, in d3dumddi.h, you can see how some parts of the D3DDDI_DEVICEFUNCS struct are conditionally defined depending on D3D_UMD_INTERFACE_VERSION, but since this struct is filled by the driver, it's the driver's D3D_UMD_INTERFACE_VERSION that matters. You should not try to hook past what the driver supports, since you might be reading uninitialized or inaccessible memory. In other words, the size of the D3DDDI_DEVICEFUNCS struct returned by the driver depends on its own D3D_UMD_INTERFACE_VERSION, even if your own build environment supports a newer version. The driver's interface version is returned in OpenAdapter's pOpenData->DriverVersion.

If I have to map the word "and" in a logical and condition I get no calls that match the end of all three conditions. So I put the conditions in logical or and traced which element triggered the condition.

I meant all 3 conditions with "and", yes. If Format is D3DDDIFMT_VERTEXDATA, then Flags.VertexBuffer must be set and vice versa. It would be invalid not to set both at the same time. In my logs, it looks correct. DDrawCompat actually only checks for the flag, not the format. Make sure you're not mixing up Flags with Flags2, although the latter doesn't even have a VertexBuffer member, but maybe you're checking the full DWORD value, and not the member variable called "VertexBuffer" in the bitfield struct?

The system memory pool is partially my bad. This game doesn't use vertex buffers at all, but Direct3D creates some implicit vertex buffers during CreateDevice, and sometimes copies the user-memory vertex data there, instead of using pfnSetStreamSourceUm directly. And then of course it uses pfnSetStreamSource with those implicitly created vertex buffers.

The trick is that the driver initially tries to create these implicit buffers in video memory, and only falls back to system memory if it fails. Years ago, I observed some performance issues with the video memory versions, so I added code to intentionally fail their creation, by returning with E_FAIL from pfnCreateResource(2), but the actual error code probably doesn't matter much. A simple way to detect these implicit buffers is that apart from Flags.VertexBuffer, also Flags.MightDrawFromLocked will be set. Only these implicit buffers should ever use the latter flag. So check for those two flags and if the Pool is not system memory, return with some error.

Here are the logs from DDrawCompat for the creation of the first implicit vertex buffer, both the blocked video memory case and then the system memory fallback (excluding some uninteresting inner logs):

2570 23:12:49.894   > IDirectDraw4Vtbl::CreateSurface(&06B68170, *{0x5,0,65536,0,0,0,0,0,null,{0x0,0x0,0,0x0,0x0,0x0,0x0},{0x804000,0x0,0x0,0x0},0}, *null, null)
2570 23:12:49.894     > _D3DDDI_DEVICEFUNCS::pfnCreateResource2(&0799F000, *{D3DDDIFMT_VERTEXDATA,D3DDDIPOOL_VIDEOMEMORY,0,0,[{65536,1,0,null,0,0}],1,0,0,0,{0,0},&07C1E4C0,0x2080000,1,0x0})
2570 23:12:49.894     < _D3DDDI_DEVICEFUNCS::pfnCreateResource2(&0799F000, *{D3DDDIFMT_VERTEXDATA,D3DDDIPOOL_VIDEOMEMORY,0,0,[{65536,1,0,null,0,0}],1,0,0,0,{0,0},&07C1E4C0,0x2080000,1,0x0}) = 80004005
2570 23:12:49.894   < IDirectDraw4Vtbl::CreateSurface(&06B68170, *{0x5,0,65536,0,0,0,0,0,null,{0x0,0x0,0,0x0,0x0,0x0,0x0},{0x804000,0x0,0x0,0x0},0}, *null, null) = 80004005
2570 23:12:49.894   > IDirectDraw4Vtbl::CreateSurface(&06B68170, *{0x5,0,65536,0,0,0,0,0,null,{0x0,0x0,0,0x0,0x0,0x0,0x0},{0x800800,0x0,0x0,0x0},0}, *null, null)
2570 23:12:49.894     > _D3DDDI_DEVICEFUNCS::pfnCreateResource2(&0799F000, *{D3DDDIFMT_VERTEXDATA,D3DDDIPOOL_SYSTEMMEM,0,0,[{65536,1,0,&07C497E0,65536,0}],1,0,0,0,{0,0},&07C1E060,0x2080000,1,0x0})
2570 23:12:49.894     < _D3DDDI_DEVICEFUNCS::pfnCreateResource2(&0799F000, *{D3DDDIFMT_VERTEXDATA,D3DDDIPOOL_SYSTEMMEM,0,0,[{65536,1,0,&07C497E0,65536,0}],1,0,0,0,{0,0},&07C06C50,0x2080000,1,0x0}) = 0
2570 23:12:49.894   < IDirectDraw4Vtbl::CreateSurface(&06B68170, *{0x5,0,65536,0,0,0,0,0,null,{0x0,0x0,0,0x0,0x0,0x0,0x0},{0x800800,0x0,0x0,0x0},0}, *&07C6E7C8, null) = 0

ghotik commented 2 weeks ago

Hi again. I slowly started to hook all necessary methods and added some logging. The part that inhibits vertices on video memory seems to work, the selected resources are now built in system memory. Then I added an association table, and here I'm going to have some troubles, maybe I got your instructions wrong. The table is enumerating couple of handles and memory pointers. In Thief2 I get vertices buffers from pfnCreateResource2. Then the handle is matched with the pData->hVertexBuffer field in pfnSetStreamSource but now I don't know how to associate the collected information with the pfnDrawPrimitive operation. pfnDrawPrimitive doesn't get the vertex buffer handle (it only receives the device handle that never matches with the vertex handle) and doesn't receive the vertex buffer address, so it seems that I miss a necessary link ... please, help!! P.s. I say something obvious, but the game is creating several vertex buffers, so I should find a way to pick the correct one ...

edit Maybe I got the flaw in my reasoning: the pfnCreateResource and pfnSetStreamSource are instanced multiple times, but they seem to always refer to the same hVertexBuffer value and the same memory address. So, maybe there's ONLY ONE vertex buffer and the pfnDrawPrimitive should refer to that single value?

Now some more: in order to trim the z values to 1.0 maximum there should be no problems, the operation is quick and safe enough to be repeated endlessly with no big impact. But I'm starting to grow a wicked idea: what if DxWnd could increase all x, y coordinates by a given factor but just once? Could that be a sort of magnifier or zooming feature? Maybe saving the original values somewhere so that the zooming could be chosen at will. But of course I have to fix the z problem first, then maybe I'll try to develop this other idea.

narzoul commented 2 weeks ago

You need to use whatever vertex source was last specified via pfnSetStreamSource(Um). If the last such call was with a vertex buffer (pfnSetStreamSource case), then use that vertex buffer later at the time of pfnDrawPrimitive. If it was a user-mode memory pointer (pfnSetStreamSourceUm case), then use that pointer for pfnDrawPrimitive. Most driver functions other than pfnDrawPrimitive just set up the device state for drawing, so the driver uses the last received state values (be it the vertex source, render state, texture stage state, etc.).

Yes, you can modify x, y values to achieve zooming, but I'm not sure how you'd ensure that the values don't change again in the original vertex source, since the source is in system memory that the CPU could modify any time, without calling any driver functions. So it's best not to cache the results and just reapply the modification always to some internal copy of the vertex data. This is what DDrawCompat does. Of course, then you have to change the vertex source (with pfnSetStreamSource(Um)) to your own internal, modified copy before calling the real pfnDrawPrimitive.

ghotik commented 2 weeks ago

Thank you. I think I put in practice most of your information, surely enough to add a z trimming that has no big side effects on Thief2. Surely there will be some errors, but I hope to be able to fix them. About the zooming feature, I added it as an experiment, but this brings big problems and the result doesn't seem so exciting, so I'll put that aside for the moment. Just for curiosity, I wanted to complete the logic adding the wrapper for pfnDrawIndexedPrimitive by copying and adapting what I did in pfnDrawIndexedPrimitive2, but I couldn't find where the index buffer starts in this case, there are no memory pointers in the arguments! Sadly, on my testbed the z trimming works but the stars (that is, the full set of the sky textures) are still not shown after a change in the Options > Video > Sky Detail OFF and ON again. But maybe this is another problem like a game bug. I'll ask for the results of tests on other computers where the stars never appeared in the sky.

update Maybe I know, perhaps should I hook pfnSetIndices ...

narzoul commented 2 weeks ago

Yes, pfnSetIndices(Um) sounds like the right functions. I never bothered with them because DX7 doesn't support index buffers. They were added in DX8.

BEENNath58 commented 2 weeks ago

I found a bug in the DxWnd emulation. The Stars are back in the sky, but on a Sky Detail High -> Low -> High transformation, they go missing.

This behaviour was also observed on Ivy Bridge Intel drivers (pre 2021 refresh drivers) without any wrapper used

ghotik commented 2 weeks ago

Hello. The zooming feature is starting to work: I had to copy the indices and copy back after the scaling and also add a boolean control array to avoid multiple scaling of the same vertices (if you are curious I can share the code), and now it is working quite nicely. The main purpose of this feature is to have a quick control about the completeness of the vertices handling: any distorted image surely means that some vertex has not been processed as it should, likely because it was ignored and skipped. So, for instance, this is the case of some scene in Thief 2: all the scene must come from a 3D rendering with textures, but for some reason the walls of an archivolt are not scaled like the other elements (the iron fence, the lamps ...). I'm attaching here the screenshots of the original and scaled scene. So, the idea is that probably the hooking of pnfDrawIndexedPrimitive2 and pfnDrawPrimitive doesn't process all the vertices. In my coding there is also an incomplete tracing hook of pfnDrawIndexedPrimitive and this callback is not referenced too. Do you have some idea about what could be missing? In addition, I met other situations where the vertex processing was not intercepted by this current implementation, in particular a pure D3D8 game (I don't recall the name right now ...) that seemed to ignore all my callback hooking.

original_archivolt scaled_archivolt

Assuming that you are familiar enough with DxWnd, this screenshot taken with the "Higlight textures" function shows that, at the application level, there is no difference between the scaled objects (iron fence, lamps ...) and the surrounding walls, everything is drawn through textures, so the different processing must be an inner twist in the driver logic, isn't it?

highlighed

narzoul commented 2 weeks ago

I found a bug in the DxWnd emulation. The Stars are back in the sky, but on a Sky Detail High -> Low -> High transformation, they go missing.

It's not a bug in DxWnd, it happens with DDrawCompat too. I think it's just a game engine bug. The stars reappear again when loading a save game or restarting the mission. Maybe the vertices need to be loaded when the level is loaded, and switching to low detail unloads them, but switching back to high doesn't reload them, or something like that.

So, for instance, this is the case of some scene in Thief 2: all the scene must come from a 3D rendering with textures, but for some reason the walls of an archivolt are not scaled like the other elements (the iron fence, the lamps ...).

I could reproduce the issue with DDrawCompat by applying the zoom only when the stride is 32. The gate and the lamps use a stride of 40 bytes. So I'd assume you're not keeping proper track of the stride somewhere, or not advancing to the next vertex by using the stride as the offset between vertices. Both are rendered with non-indexed vertices, so otherwise it seems to be a relatively simple case.

In addition, I met other situations where the vertex processing was not intercepted by this current implementation, in particular a pure D3D8 game (I don't recall the name right now ...) that seemed to ignore all my callback hooking.

I don't know, I never hooked D3D8, but I'd assume it to be similar to the DX7 case, except maybe it also uses the other ones of the pfnDraw(Indexed)Primitive function pairs.

There are also pfnDrawRectPatch and pfnDrawTriPatch, These also seem to have been introduced by DX8.

ghotik commented 1 week ago

Damn, you're right!

scaled_ok

ghotik commented 1 week ago

A short update to let you know just two things: 1) after some struggle, it seems that now the vertex scaling is working pretty well, though quite useless in most cases. 2) despite the "software traps" placed in the code, D3D8 seems immune to this trick, none of the callbacks mentioned above is ever triggered. Could it be that there other callbacks, or maybe they should be hooked in some different way? Just for curiosity ...

narzoul commented 1 week ago

despite the "software traps" placed in the code, D3D8 seems immune to this trick, none of the callbacks mentioned above is ever triggered. Could it be that there other callbacks, or maybe they should be hooked in some different way? Just for curiosity ...

No idea, I never tried hooking D3D8.

ghotik commented 5 days ago

I think I'm getting close to the end of this experiment. There are a few problems and questions, I don't ask for answers, just listing the issues.

1) I think I didn't understand fully the explanation about the Flexible Vertex Format. I suppose that once we find that the vertex source contains untransformed vertices we take note of it, so we should avoid making operations on the untransformed vertices on the following Draw*Primitive callbacks, . This would be fine for a Z value trimming, but it would be a problem for zooming because the zoomed picture would appear correct only if you scale all vertices.

2) as I said D3D8 seems to be avoiding this type of callbacks mechanism completely, so as other games like "Fable: the Lost Chapters".

3) another problem came from a game where I had high expectations, since the zooming would be quite useful in that case: "Project I.G.I.". Here I found a curious value of stride equal to 28 = 7 sizeof(float). In all other cases I always got a 32 = 8 sizeof(float) and the vertex addresses were successfully casted to a D3DVERTEX structure with the same size. In Project IGI the scaling makes weird effects that makes me thing that the X, Y offsets and sizes don't correspond exactly to those of a D3DVERTEX structure (among other oddities, the Y coordinate doesn't scale), but I don't know what data schema I should use in that case.

narzoul commented 5 days ago

I think I didn't understand fully the explanation about the Flexible Vertex Format. I suppose that once we find that the vertex source contains untransformed vertices we take note of it, so we should avoid making operations on the untransformed vertices on the following Draw*Primitive callbacks, . This would be fine for a Z value trimming, but it would be a problem for zooming because the zoomed picture would appear correct only if you scale all vertices.

I don't think you can do zooming like that for untransformed vertices, since the coordinates are still in model space. If you only scale X and Y, you'll stretch the object along those axes. You'd at least have to scale Z too, but this would still just make the individual objects themselves bigger, and offset them from their respective origins (in model space) in unpredictable ways. Then, the later transformations will likely misplace the objects from their intended locations, or they'll clip into each other because they're too big, etc. Read more about the world/view/projection transformations here: https://learn.microsoft.com/en-us/windows/win32/direct3d9/transforms

another problem came from a game where I had high expectations, since the zooming would be quite useful in that case: "Project I.G.I.". Here I found a curious value of stride equal to 28 = 7 sizeof(float). In all other cases I always got a 32 = 8 sizeof(float) and the vertex addresses were successfully casted to a D3DVERTEX structure with the same size. In Project IGI the scaling makes weird effects that makes me thing that the X, Y offsets and sizes don't correspond exactly to those of a D3DVERTEX structure (among other oddities, the Y coordinate doesn't scale), but I don't know what data schema I should use in that case.

It is not possible to separate the Y and Z coordinates from X. They always come after one another in the same order. Their position within a vertex is also fixed at the very beginning, so D3DVERTEX should work. In DX7, all vertex attributes have a fixed order, and you can only skip some of them, not reorder them. Refer to the DX7 SDK help files for details: Direct3D -> Direct3D Immediate Mode -> Direct3D Immediate Mode Essentials -> Vertex Formats.

I think 32 is the size of D3DTLVERTEX. 28 could be missing the specular color maybe, or something like that, assuming it's still some T&L format. But the vertex declaration created by pfnCreateVertexShaderDecl and passed to pfnSetVertexShaderDecl should tell exactly what components are present in the vertex, and at which offset.

narzoul / DDrawCompat