search

Reloaded-Project / Reloaded.Hooks

Advanced native function hooks for x86, x64. Welcome to the next level!
GNU Lesser General Public License v3.0
213 stars 33 forks source link

Hooking d3d11.Present #9

Closed daemitus closed 3 years ago

daemitus commented 3 years ago

I'm porting a project to net5 and trying to migrate off net472+easyhook. Currently, hooking d3d11.Present to render some ImGui windows is causing a crash-to-desktop on calling OriginalFunction. The data coming into the function looks to be valid as far as I can tell. I assume the issue is the calling convention, but I'm not entirely sure. If I swap out Reloaded for CoreHook, it works as expected.

This is the delegate currently being used to create the hook. 

[UnmanagedFunctionPointer(CallingConvention.ThisCall)]
private delegate IntPtr PresentDelegate(IntPtr swapChain, uint syncInterval, uint presentFlags);

Do I need to use a calling convention wrapper to make this work properly or take some sort of other additional step?

Sewer56 commented 3 years ago

As long as the address you're hooking is right, you should be good if you switch the convention to stdcall.

While it is true that IDXGISwapChain is a C++ interface and the calling convention would probably be thiscall if it was part of the compiled program; consider the fact that you are technically interfacing with DX11 through COM where the standard calling convention is generally stdcall.

For a quick cheat sheet, you can always look at the definition of CallingConventions:

https://github.com/Reloaded-Project/Reloaded.Hooks/blob/340531687e9a725dded36cbf818562f77a25f92c/source/Reloaded.Hooks.Definitions/X86/CallingConventions.cs#L21-L30

https://github.com/Reloaded-Project/Reloaded.Hooks/blob/340531687e9a725dded36cbf818562f77a25f92c/source/Reloaded.Hooks.Definitions/X86/CallingConventions.cs#L45-L58

Extra Note: I've already made a Dear ImGui overlay you could use with Reloaded. :P

Edit: And don't forget to use Reloaded's own Function attribute.

You can probably get away with something like this:

[Function(CallingConventions.Stdcall)]
public delegate IntPtr Present(IntPtr swapChainPtr, uint syncInterval, uint flags);
daemitus commented 3 years ago

Hadn't seen your ImGui impl before. Great resource. Thanks for the quick response :)

daemitus commented 3 years ago

@Sewer56 Apparently, the hook always worked just fine. It crashes due to the Discord overlay hook being already in-place on the same function. Got any ideas?

daemitus commented 3 years ago

Well one thing that's worked so far, although I'm not entirely sure if it's the smartest way to do things: hooking the discord hook (or any other hook that hooks present) instead. Following any 0xE9 or 0xFF jumps until it gets back to real code.