search

dxgldotorg / dxgl

DXGL DirectX-to-OpenGL Wrapper source code
https://dxgl.org
GNU Lesser General Public License v2.1
193 stars 16 forks source link

Fix memory corruption caused by dwSize override #3

Closed realmonster closed 4 years ago

realmonster commented 4 years ago

It's normal to use same struct twice and more times. Game that I tried to run with DXGL was crashing after few common API calls. From other hand it works fine with standard ddraw. The value of EIP at crash out loudly tells us that crash is caused by stack corruption. So I verified those few API calls and it turns out that GetDisplayMode was overriding dwSize with value bigger than it was set, and then GetSurfaceDesc was filling DDSURFACEDESC according to the new dwSize causing stack corruption.

First, I made hot fix, and it worked. Then, I made little test: https://gist.github.com/realmonster/0af97e2d1e040182b7751558469f409e

And after fixing all issues it show, new version for some reason even fixed more bugs in the game. I have no idea where they come from but after making this test pass without issues, the game under DXGL I test works with it even better than before fix.

dxgldotorg commented 4 years ago

Thanks, I'll implement it as soon as I give some legacy OpenGL API calls the axe; it should not at all conflict with my next changes.

dxgldotorg commented 3 years ago

It appears the same issue is plaguing IDirect3DDevice2:EnumTextureFormats.