crosire / d3d8to9

A D3D8 pseudo-driver which converts API calls and bytecode shaders to equivalent D3D9 ones.
BSD 2-Clause "Simplified" License
892 stars 80 forks source link

CreateVertexBuffer always replaces D3DPOOL_MANAGED with D3DPOOL_DEFAULT #1

Closed michael-fadely closed 7 years ago

michael-fadely commented 7 years ago

In 8678d6f5f867873cca850b5b679d5804c31b8ca9 a workaround was added for erroneous lock flags by changing D3DPOOL_MANAGED to D3DPOOL_DEFAULT. This poses a problem for games that use D3DPOOL_MANAGED to avoid the hassle of re-initializing data (i.e on resolution change).

In native D3D8 games that use managed and even incorrect lock flags, the program will continue running fine. Is this not the case in D3D9? Could it perhaps be modified to only use the default pool if D3DUSAGE_DYNAMIC is already present?

crosire commented 7 years ago

According to both the D3D8 and D3D9 documentation, D3DLOCK_DISCARD may only be used on vertex buffers created with the D3DUSAGE_DYNAMIC flag. However it is too invalid to create a D3DPOOL_MANAGED buffer with that flag. It is valid only with D3DPOOL_DEFAULT (this too is true for both D3D8 and D3D9). I cannot remember which game it was that attempted to use the discard flag on a managed buffer, but it refused to launch unless the workaround was applied, which is why it was added. Probably because the call returned an error (which should be the case in native D3D8 as well, so I have no idea why it works there). The game did not pass D3DUSAGE_DYNAMIC either.

A better idea might be to remove the D3DLOCK_DISCARD flag from the lock call if the buffer usage does not match D3DPOOL_DEFAULT. That would invoke different behaviour, but at least should allow the call to succeed. And since it is the application's fault for using invalid flags, I'd expect undefined behaviour in native D3D8 here as well.

crosire commented 7 years ago

I pushed a proposition to the fix_issue_1 branch. Please check if that works for you.

michael-fadely commented 7 years ago

6d957118764329b68caf3c911539c08c6ff18a8a is causing the game in question (original 2004 Sonic Adventure DX PC) to crash sometime after creating IDirect3DDevice8, so I can't say. Somehow it's returning from a function call to some non-executable memory. It's calling IDirect3D8::GetDeviceCaps followed by IDirect3D8::CreateDevice. If the device is still null by this point, it tries again with different parameters.

crosire commented 7 years ago

That commit changed exactly nothing, I only moved some code around ...

michael-fadely commented 7 years ago

That's what I figured, which made things all the more confusing. But alas, reverting to the revision right before that fixes it. I have no idea what's going on there...

michael-fadely commented 7 years ago

Found the problem here. This should be sizeof(output) not sizeof(input). The pointer the game passes in is to a structure on the stack, and this stomps all over it and messes up the return address.

(Edit: and yeah, the intended change did fix the vertex buffer issue)

crosire commented 7 years ago

Ohhhhh .... So I did change one thing and of course had to mistake that one thing. Great. Anyway. Will fix that and merge it back into master then.