Dasaav-dsv / CallHook

An x86-64 function call hooking library.
MIT License
9 stars 4 forks source link

Failed to allocate memory, size 200: no suitable memory region found #1

Closed Acedia0 closed 7 months ago

Acedia0 commented 7 months ago

Hey, I just tested your hook library by slightly modifying the example. However, I keep getting the same error regardless of the type of hook I use:

callhookexample\log\CallHook: Allocating 200 bytes near module x2.exe callhookexample\log\CallHook: Failed to allocate memory, size 200: no suitable memory region found callhookexample\log\CallHook: Error: failed to hook call at 00000000347E21DD - unable to allocate memory.

Does anyone know what could be causing this? It's strange that the same memory size is allocated for different types of hooks. Thanks in advance.

Acedia0 commented 7 months ago

I got it to work by changing the end of the moduleAlloc function in Alloc.h :

void* moduleAlloc(std::size_t cb, LPCSTR lpModuleName = NULL) { auto& logger = Logger::get(); std::string moduleName; // Get the name of the base module if a module name wasn't already provided. if (lpModuleName == NULL) { auto moduleNameShort = std::make_unique<char[]>(MAX_PATH); moduleName = Logger::getCurrentModuleName(); if (errno_t error = _splitpath_s(moduleName.c_str(), NULL, 0, NULL, 0, moduleNameShort.get(), MAX_PATH, NULL, 0)) { logger.log("Failed to allocate memory: unable to get main module name; error %d", error); return nullptr; } else { moduleName = std::string(moduleNameShort.get()) + ".exe"; } } else { moduleName = std::string(lpModuleName); } logger.log("Allocating %d bytes near module %s", cb, moduleName.c_str()); MODULEINFO modInfo{}; MEMORY_BASIC_INFORMATION memInfo{}; if (HMODULE hModule = GetModuleHandleA(moduleName.c_str())) { if (!GetModuleInformation(GetCurrentProcess(), hModule, &modInfo, sizeof(MODULEINFO))) { logger.log("Failed to allocate memory: unable to get module information; error %d", GetLastError()); return nullptr; } } else { logger.log("Failed to allocate memory: unable to get module handle; error %d", GetLastError()); return nullptr; } uintptr_t moduleStart = reinterpret_cast(modInfo.lpBaseOfDll); uintptr_t moduleEnd = moduleStart + modInfo.SizeOfImage;

// Establish the base address to search for suitable memory from, it must be within an INT_MAX range from any point in the module.
void* baseAddress = nullptr;

// Rechercher une région de mémoire appropriée dans toute la mémoire du processus
baseAddress = nullptr;
while (VirtualQuery(baseAddress, &memInfo, sizeof(memInfo))) {
    // Vérifier si la région de mémoire est libre et assez grande
    if (memInfo.State == MEM_FREE && memInfo.RegionSize >= cb) {
        // Allouer la mémoire dans cette région
        void* alloc = VirtualAlloc(memInfo.BaseAddress, cb, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);
        if (alloc) {
            logger.log("Allocated %d bytes at %p", cb, alloc);
            return alloc;
        }
    }
    // Passer à la région de mémoire suivante
    baseAddress = static_cast<char*>(memInfo.BaseAddress) + memInfo.RegionSize;
}

// Aucune région de mémoire appropriée trouvée
logger.log("Failed to allocate memory, size %d: no suitable memory region found", cb);
return nullptr;

}

Dasaav-dsv commented 7 months ago

I see, it's weird that the call to VirtualAlloc fails here. The piece of code you removed makes sure that the memory region allocated for the hook is as close as possible to the end of the checked memory region. This is done by subtracting the size of the allocation cb from the pointer to the end of the region end:

void* alloc = reinterpret_cast<void*>(end <= moduleStart ? end - cb : begin);

This has never failed for me (or the many others who have used my mods - for example, memory allocations in ELDEN RING for hooks will often happen below the base address of the application image). According to Microsoft, the address given to VirtualAlloc will be rounded down to the nearest page/allocation granularity boundary. If you have any ideas why it fails in the environment you run it in (please provide some details like your OS, arch etc.), please let me know so I can fix it.

Acedia0 commented 7 months ago

I have no idea where the problem might come from, my case is quite particular. I'm trying to perform a hook on a game (Windows x64) that has the GameGuard anti-cheat and that, in addition, it has some anti-debugging techniques, (it doesn't have a .text or rdata section) so it's impossible to use IDA and also impossible to use your function CallHook::CallMap callMap{} since it depends on the .text section. I had to create a custom function that searches for corresponding calls throughout the process memory.

I could do more debugging to find out if all the calculations are going well, but I really lost the desire to do it given all the hook techniques I've used get detected... Memory read/write, DLL injection, or thread creation are not detected, but using a hook always is. An analysis of the Impact on the Program's Operation (slowing down or otherwise...) may have been implemented... Because writing a trampoline in memory poses no problem, but redirecting a call or replacing the beginning of the hooked function with a call/jump does.

Anyway, I find myself having to analyze the stack of a function to retrieve an argument (LuaState pointer in Lua) from a DLL, so no need for a hook. I just hope that no verification of the number of function calls is done...

Dasaav-dsv commented 7 months ago

Oh I see, yes that is pretty specific. It's not an AC I'm familiar with the workings of. Anyhow, seems like the issue is in the measures in place and not in my code. If you ever figure out which part made it fail (and why your fix worked) let me know, I am curious. Good luck with your project!

Acedia0 commented 7 months ago

Thank you, and yes, I'll let you know if I ever figure out where the issue is coming from.

By the way, I've encountered some strange crashes when attempting an Individual call unhooking (deleting hook):

callhookexample\log\CallHook: Successfully initialized CallHook callhookexample\log\CallHook: Hooking call at 0000000033219DF0 - allocating hook memory callhookexample\log\CallHook: Allocating 200 bytes near module x2.exe callhookexample\log\CallHook: Allocated 200 bytes at 00000000022B0000 callhookexample\log\CallHook: Successfully hooked call at 0000000033219DF0 callhookexample\log\CallHook: Unhooking call at 0000000033219DF0 callhookexample\log\CallHook: Error when patching call at 0000000033219DF0, memory is inaccessible callhookexample\log\CallHook: Successfully unhooked call at 0000000033219DF0

After some debugging, I found that the current thread didn't have permissions to write to that location (error 5 from VirtualProtect) even though it had permissions to write the hook. So, the anticheat must have locked the memory location at some point.