stevemk14ebr / PolyHook_2_0

C++20, x86/x64 Hooking Libary v2.0
MIT License
1.6k stars 226 forks source link

Crash on VFuncSwapHook/VTableSwapHook Constructor #161

Closed Waffle1434 closed 1 year ago

Waffle1434 commented 1 year ago

I am absolutely losing my mind. I tried to hook a DirectX 11 function based off https://github.com/stevemk14ebr/PolyHook_2_0/blob/master/UnitTests/windows/TestVFuncSwapHook.cpp

Before constructing VFuncSwapHook hook, redir map is valid: image The moment it enters the constructor, redir which should have been passed by reference is completely invalid: image The program crashes on m_redirectMap(redirectMap) as it appears to be trying to construct a map based on an invalid map. I am running in x64 debug mode, no optimizations should be applied here.

#include <d3d11_1.h>
#pragma comment(lib, "d3d11.lib")

#include "polyhook2/Detour/x64Detour.hpp"
#include "polyhook2/Virtuals/VFuncSwapHook.hpp"
#include "polyhook2/Virtuals/VTableSwapHook.hpp"
#include "polyhook2/StackCanary.hpp"
#pragma comment(lib, "PolyHook_2.lib")

PLH::VFuncMap origVFuncs2;
HOOK_CALLBACK(&ID3D11DeviceContext::IASetIndexBuffer, hkVirtTest, {
   PLH::StackCanary canary;
   MessageBoxA(NULL, "Hooked", "Test", 0);
   return ((hkVirtNoParams2_t)origVFuncs2.at(1))(_args...);
});

int main() {
    DXGI_SWAP_CHAIN_DESC sd;
    ZeroMemory(&sd, sizeof(sd));
    sd.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
    sd.BufferCount = 2;
    sd.BufferDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM_SRGB;
    sd.OutputWindow = GetForegroundWindow();
    sd.SampleDesc.Count = 1;
    sd.Windowed = TRUE;
    sd.SwapEffect = DXGI_SWAP_EFFECT_DISCARD;
    sd.Flags = 0;

    IDXGISwapChain* swapchain;
    ID3D11Device* device;
    ID3D11DeviceContext* dev_context;

    const D3D_FEATURE_LEVEL feature_levels[] = { D3D_FEATURE_LEVEL_11_0 };
    HRESULT res = D3D11CreateDeviceAndSwapChain(NULL, D3D_DRIVER_TYPE_HARDWARE, NULL, 0, feature_levels, ARRAYSIZE(feature_levels), D3D11_SDK_VERSION, &sd, &swapchain, &device, NULL, &dev_context);
    if (res == S_OK) {
        PLH::StackCanary canary;
        PLH::VFuncMap redir = { {(uint16_t)1, (uint64_t)hkVirtTest} };
// Crash Here =================================================================================================
        PLH::VFuncSwapHook hook((char*)(dev_context), redir, &origVFuncs2); // Crash inside the constructor, in the "main" scope "redir" is properly initialized and has a size of 1. INSIDE the constructor however, "redir" is not passed by reference correctly and has an invalid size of 
        //PLH::VTableSwapHook hook((char*)dev_context, redir, &origVFuncs2);
        assert(hook.hook());

        swapchain->Release();
        device->Release();
        dev_context->Release();
    }

    return 0;
}

BOOL APIENTRY DllMain(HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved) {
   switch (ul_reason_for_call) {
      case DLL_PROCESS_ATTACH:
         MessageBoxA(NULL, "Attach Debugger", "Test", 0);
         CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)main, NULL, 0, NULL);// Create to get out of remote thread
         break;
      case DLL_PROCESS_DETACH:
      case DLL_THREAD_ATTACH:
      case DLL_THREAD_DETACH:
         break;
   }
   return TRUE;
}
stevemk14ebr commented 1 year ago

Please debug at the assembly level to determine the crash. It's not uncommon for the debugger to not be able to read object layouts in memory so seeing errors trying to read the map could be unrelated. Also ensure the headers you're using match the Library version you are linking, if you're using vcpkg for example, ensure you're not accidentally including headers from a clone of this master as the versions differ.

First thing I'd try is making your map global, it's taken by reference so could be being destructed in the local scope, somehow.