search

Sewer56 / Reloaded.Imgui.Hook

A simple personal use utility library that can be used to inject Dear Imgui into the current process.
MIT License
17 stars 5 forks source link

Access violation on resizing swapchain (DX11) #7

Closed NotNite closed 1 month ago

NotNite commented 1 month ago

This happens on Rabbit and Steel, but it might happen on any game using modern GameMaker (previously called GameMaker Studio 2).

Resizing swap chain...
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Repeat 2 times:
--------------------------------
   at DearImguiSharp.ImGui+__Internal.ImGuiImplDX11InvalidateDeviceObjects()
--------------------------------
   at Reloaded.Imgui.Hook.Direct3D11.ImguiHookDx11.ResizeBuffersImpl(IntPtr, UInt32, UInt32, UInt32, SharpDX.DXGI.Format, UInt32)
   at Reloaded.Imgui.Hook.Direct3D11.ImguiHookDx11.ResizeBuffersImplStatic(IntPtr, UInt32, UInt32, UInt32, SharpDX.DXGI.Format, UInt32)

Unsure how to get a better stacktrace, I tried attaching in Rider but it was useless and x64dbg didn't show anything of interest. The game uses a GameMaker extension that adds a unique window frame called gameframe - dunno if that matters, but including just in case.

Sewer56 commented 1 month ago

Can't really get a better stack trace. Unfortunately. It seems to be crashing in native Dear ImGui code, so most likely you'd need to:

Something of that sort anyway. That's how I'd start.

NotNite commented 1 month ago

I ended up figuring it out - I don't know if this behavior is valid across other programs so I'll just leave this here for now: https://github.com/NotNite/Reloaded.Imgui.Hook/commit/df4234fc9c50b3e26d8bbd689e4051750d035890

The game resizes extremely early for some reason, and this causes ResizeBuffers to enter before the backend data is initialized (somehow, I haven't looked into it too hard). ImGui does not null check the backend data, leading to the access violation. This is probably invalid behavior from the ImGui side, so I guess it should be reported there (assuming they even care about issues you could only stumble into from function hooking).

Thanks for the quick response (and Reloaded)!

Sewer56 commented 1 month ago

Is it possible to make this check by checking on _initialize field? Might be more performant.

(Do also consider PR-ing it 😉)

NotNite commented 1 month ago

Seems like that works - I'll make a PR.