VFuncSwap doesn't work on x64?

[zezba9000]

So am trying to hook a Virtual method but its not work on 64 builds. I get an exception when trying to delete the "delete shimTrackedDevice;" Also the "thisptr" is invalid My system is Win10 x64, Visual Studio 2017.

referencing: https://www.codeproject.com/articles/1100579/polyhook-the-cplusplus-x-x-hooking-library

// ----------------------------------------------------------
// hooks
// ----------------------------------------------------------
void** trackedDeviceVTable = nullptr;
constexpr int trackedDeviceVTableIndex_GetPose_Index = 5;
typedef DriverPose_t(__thiscall *GetPose_Org)(ITrackedDeviceServerDriver* thisptr);
GetPose_Org GetPose_Ptr = nullptr;

float x = 0;
ITrackedDeviceServerDriver* thisptr__ = nullptr;
static DriverPose_t __fastcall GetPose_Hook(ITrackedDeviceServerDriver* thisptr)//, int edx)
{
    if (thisptr__ == thisptr) cout << "Valid This Ptr!!!" << endl;
    else cout << "Invalid This Ptr!!!" << endl;
    return GetPose_Ptr(thisptr);
}

PLH::VFuncSwap* VFuncSwap_Ex = new PLH::VFuncSwap;
void HookDriver(ITrackedDeviceServerDriver* driver, bool usePolyHook)
{
    thisptr__ = driver;

    if (usePolyHook)
    {
        VFuncSwap_Ex->SetupHook(*(BYTE***)driver, trackedDeviceVTableIndex_GetPose_Index, (BYTE*)&GetPose_Hook);
        VFuncSwap_Ex->Hook();
        GetPose_Ptr = VFuncSwap_Ex->GetOriginal<GetPose_Org>();
    }
    else
    {
        trackedDeviceVTable = *(void***)(driver);
        #ifdef _32BIT
        MEMORY_BASIC_INFORMATION32 trackedDeviceMBI;
        VirtualQuery((LPCVOID)trackedDeviceVTable, (PMEMORY_BASIC_INFORMATION)&trackedDeviceMBI, sizeof(MEMORY_BASIC_INFORMATION32));
        #else
        MEMORY_BASIC_INFORMATION64 trackedDeviceMBI;
        VirtualQuery((LPCVOID)trackedDeviceVTable, (PMEMORY_BASIC_INFORMATION)&trackedDeviceMBI, sizeof(MEMORY_BASIC_INFORMATION64));
        #endif
        VirtualProtect((LPVOID)trackedDeviceMBI.BaseAddress, trackedDeviceMBI.RegionSize, PAGE_EXECUTE_READWRITE, &trackedDeviceMBI.Protect);// unlock
        GetPose_Ptr = (GetPose_Org)trackedDeviceVTable[trackedDeviceVTableIndex_GetPose_Index];
        trackedDeviceVTable[trackedDeviceVTableIndex_GetPose_Index] = &GetPose_Hook;// Hook!
        VirtualProtect((LPVOID)trackedDeviceMBI.BaseAddress, trackedDeviceMBI.RegionSize, trackedDeviceMBI.Protect, &trackedDeviceMBI.Protect);// lock
    }
}

static ShimTrackedDevice* shimTrackedDevice = new ShimTrackedDevice();
int main()
{
    cout << "Call before hook..." << endl;
    auto result = shimTrackedDevice->GetPose();

    cout << "Hooking..." << endl;
    HookDriver(shimTrackedDevice, true);

    cout << "Calling Hooked..." << endl;
    result = shimTrackedDevice->GetPose();

    cout << "Deleting..." << endl;
    delete shimTrackedDevice;
    shimTrackedDevice = nullptr;

    cout << "Finished!" << endl;
    return 0;
}

[zezba9000]

Return value looks like:

struct DriverPose_t
{
    double poseTimeOffset;
    HmdQuaternion_t qWorldFromDriverRotation;
    double vecWorldFromDriverTranslation[3];
    HmdQuaternion_t qDriverFromHeadRotation;
    double vecDriverFromHeadTranslation[3];
    double vecPosition[3];
    double vecVelocity[3];
    double vecAcceleration[3];
    HmdQuaternion_t qRotation;
    double vecAngularVelocity[3];
    double vecAngularAcceleration[3];
    ETrackingResult result;
    bool poseIsValid;
    bool willDriftInYaw;
    bool shouldApplyHeadModel;
    bool deviceIsConnected;
};

[stevemk14ebr]

assuming you have the correct vtable and your own implementation of the vtable hook works then i don't know why it fails. Look at my code the implementations are identical. You will have to debug yourself, i cannot debug your target. Define doesn't work, fails to hook or what?

[zezba9000]

your own implementation of the vtable hook works

-- It doesn't on x64 but does on x86. Its why I'm trying to use PolyHook. x64 isn't hooking correctly.

So to be more clear hooking a method that returns the "DriverPose_t" on x64 doesn't work correctly...

What does work on x86 (32 bit) [in short everything works]:

• PolyHook: Fully hooks and works just fine.
• My VirtualProtect Implementation: Fully hooks and works just fine.

What doesn't work on x64 (64 bit) [in short partially works]:

• PolyHook: Hooks but the "this ptr" passed into the GetPose_Hook is wrong.
• My VirtualProtect Implementation: Hooks but the "this ptr" passed into the GetPose_Hook is wrong.

So on x64 the hooked "GetPose_Hook" gets called. However the "ITrackedDeviceServerDriver* thisptr" value is invalid.

I have also tried to hook with the "VFuncDetour" object and have the same issue.

VFuncDetour_Ex->SetupHook(*(BYTE***)driver, 5, (BYTE*)&GetPose_Hook); //can cast to byte* to
VFuncDetour_Ex->Hook();
GetPose_Ptr = VFuncDetour_Ex->GetOriginal<GetPose_Org>();

FYI: with the "VFuncDetour" method I get vector "foreach" issues and need to modify the code like so:

class ThreadManager
        {
        public:
            void SuspendThreads()
            {
                UpdateThreadList(GetCurrentThreadId());
                /*for (ThreadHandle& ThreadInstance : m_SuspendedThreads)
                {
                    ThreadInstance.ToggleSuspend(true);
                }*/
                for (int i = 0; i != m_SuspendedThreads.size(); ++i)
                {
                    m_SuspendedThreads[i].ToggleSuspend(true);
                }
            }

            void ResumeThreads()
            {
                /*for (ThreadHandle& ThreadInstance : m_SuspendedThreads)
                {
                    ThreadInstance.ToggleSuspend(false);
                }*/
                for (int i = 0; i != m_SuspendedThreads.size(); ++i)
                {
                    m_SuspendedThreads[i].ToggleSuspend(false);
                }
            }

[stevemk14ebr]

your typedef for the function is wrong on x64 then. You had to modify the foreach because your compiler doesn't support c++11, this is not recommended for polyhook, upgrade to at least vs2015.

[zezba9000]

I'm using VS 2017 fully up to date. I get error: "vector iterator not dereferencable" on line "for (ThreadHandle& ThreadInstance : m_SuspendedThreads)" in the "void ResumeThreads()" method

[zezba9000]

I think this issue happens because the returned value is bigger then 64 bytes. All hooked methods work unless they return more than the register size.

[stevemk14ebr]

I'm in the process of a full re-write I will be deving on vs2017 so I will investigate foreach problem and avoid it in next release. As for your issue, it is an issue with your typedef, since this is not an issue with polyhook I am going to close this issue. I suggest you debug your target and figure out how it uses the registers and make sure your typedef is appropriate. If you have issues make a thread on unknowncheats.me (after you do research) and the guys there can explain typedefs to you in detail.

[zezba9000]

If my typedef was wrong it shouldn't work on x86 then. Its a standard function ptr typedef. The code compiles fine in both VS2015 and VS2017 (thats NOT the issue). Its NOT a compiler issue, its a runtime issue.

If I send you $200 via PayPal and a very simple project illustrating the problem in VS2015 would you be willing to take a look? I'm in a crunch to get it working and calling conventions aren't my specialty.

[stevemk14ebr]

I am aware the issue is a runtime issue, it is because of your typedef as i said. Different calling conventions exist between x86 and x64, also different optimization can happen. In this case i think return value optimization is occuring, which means instead of returning the struct by making a copy, they function is instead changed to pass a hidden parameter. https://en.wikipedia.org/wiki/Return_value_optimization

Try this for x64 (i added an extra void and removed static). I can't do more unless i see the disassembly of the function you are hooking, this is a guess as it is typedef DriverPose_t(__thiscall GetPose_Org)(void HiddenNVORet, ITrackedDeviceServerDriver* thisptr);

DriverPose_t __fastcall GetPose_Hook(void HiddenNVORet,ITrackedDeviceServerDriver* thisptr)

I have a donation button on project home page. You can donate if you wish, i will help as much as I can for free, i'm not going to hold you hostage.

[zezba9000]

Thats super close. "HiddenNVORet" now points to a valid this obj ptr.

This is the method before being hooked:

    auto result = shimTrackedDevice->GetPose();
00007FF75FAB43D3  mov         rax,qword ptr [shimTrackedDevice (07FF75FE630A8h)]  
00007FF75FAB43DA  mov         rax,qword ptr [rax]  
00007FF75FAB43DD  lea         rdx,[result]  
00007FF75FAB43E1  mov         rcx,qword ptr [shimTrackedDevice (07FF75FE630A8h)]  
00007FF75FAB43E8  call        qword ptr [rax+28h] 

This is the method after being hooked

    result = shimTrackedDevice->GetPose();
00007FF60634443F  mov         rax,qword ptr [shimTrackedDevice (07FF6066F30A8h)]  
00007FF606344446  mov         rax,qword ptr [rax]  
00007FF606344449  lea         rdx,[rbp+2D0h]  
00007FF606344450  mov         rcx,qword ptr [shimTrackedDevice (07FF6066F30A8h)]  
00007FF606344457  call        qword ptr [rax+28h]  
00007FF60634445A  lea         rcx,[result]  
00007FF60634445E  mov         rdi,rcx  
00007FF606344461  mov         rsi,rax  
00007FF606344464  mov         ecx,118h  
00007FF606344469  rep movs    byte ptr [rdi],byte ptr [rsi]  

When the delete method is call it has the error:

HEAP CORRUPTION DETECTED: after Normal block (#163) at 0x000002AA04F157B0.
CRT detected that the application wrote to memory after end of heap buffer.

ASM:

    delete shimTrackedDevice;
00007FF60634448E  mov         rax,qword ptr [shimTrackedDevice (07FF6066F30A8h)]  
00007FF606344495  mov         qword ptr [rbp+408h],rax  
00007FF60634449C  mov         edx,8  
00007FF6063444A1  mov         rcx,qword ptr [rbp+408h]  
00007FF6063444A8  call        operator delete (07FF60632BDC5h)  

Code call overview:

static ShimTrackedDevice* shimTrackedDevice = new ShimTrackedDevice();
int main()
{
    cout << "Call before hook..." << endl;
    auto result = shimTrackedDevice->GetPose();

    cout << "Hooking..." << endl;
    HookDriver(shimTrackedDevice, true);

    cout << "Calling Hooked..." << endl;
    result = shimTrackedDevice->GetPose();

    cout << "Deleting..." << endl;
    delete shimTrackedDevice;
    shimTrackedDevice = nullptr;

    cout << "Finished!" << endl;
    return 0;
}

[stevemk14ebr]

Post disassembly of the shimTrackedDevice->GetPose(); where it is called

[zezba9000]

Ok you're solution worked, I forgot to change the typedef.

So "typedef DriverPose_t(__thiscall GetPose_Org)(void HiddenNVORet, ITrackedDeviceServerDriver thisptr);" works!!!

I'll be donating soon, tnx for your help on this! Will have to spend some more time going over why this solution works in asm. Let me know if you would still like disassembly for any of this to help with your API.