fancycode / MemoryModule

Library to load a DLL from memory.
http://www.joachim-bauch.de/tutorials/loading-a-dll-from-memory/
Mozilla Public License 2.0
2.74k stars 750 forks source link

64-bit DLL compiled by G++ failed to load. #108

Open trungnt2910 opened 2 years ago

trungnt2910 commented 2 years ago

I have a 64-bit DLL, with this source code:

#include <iostream>

extern "C"
{
    __declspec(dllexport) __cdecl void Greet()
    {
        std::cout << "Hello World!" << std::endl;
    }

    __declspec(dllexport) __cdecl int addNumbers(int a, int b)
    {
        std::cout << "Adding " << a << " and " << b << "..." << std::endl;
        return a + b;
    }
}

I compiled this using g++, downloadable from here.

This is my g++ version g++ (MinGW-W64 x86_64-posix-seh, built by Brecht Sanders) 11.1.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

This is the command I used:

g++ SampleDll.cpp -o SampleDLL.dll --shared 

I. Attempt to load from file.

I have a C source file (always compiled using g++ Test.c MemoryModule.c -o Test.exe -D_WIN64) and copied the whole LoadFromFile function from the demo to my source file:

The source ```C++ void LoadFromFile(void) { addNumberProc addNumber; HRSRC resourceInfo; DWORD resourceSize; LPVOID resourceData; TCHAR buffer[100]; HINSTANCE handle = LoadLibrary(DLL_FILE); if (handle == NULL) return; addNumber = (addNumberProc)GetProcAddress(handle, "addNumbers"); _tprintf(_T("From file: %d\n"), addNumber(1, 2)); resourceInfo = FindResource(handle, MAKEINTRESOURCE(VS_VERSION_INFO), RT_VERSION); _tprintf(_T("FindResource returned 0x%p\n"), resourceInfo); resourceSize = SizeofResource(handle, resourceInfo); resourceData = LoadResource(handle, resourceInfo); _tprintf(_T("Resource data: %ld bytes at 0x%p\n"), resourceSize, resourceData); LoadString(handle, 1, buffer, sizeof(buffer)); _tprintf(_T("String1: %s\n"), buffer); LoadString(handle, 20, buffer, sizeof(buffer)); _tprintf(_T("String2: %s\n"), buffer); FreeLibrary(handle); } ```

It loads properly:

Adding 1 and 2...
From file: 3
FindResource returned 0x0000000000000000
Resource data: 0 bytes at 0x0000000000000000
String1:
String2: 

II. Attempt to load from Memory

I also copied the LoadFromMemory and the ReadLibrary function:

The source ```C++ void* ReadLibrary(size_t* pSize) { size_t read; void* result; FILE* fp; fp = _tfopen(DLL_FILE, _T("rb")); if (fp == NULL) { _tprintf(_T("Can't open DLL file \"%s\"."), DLL_FILE); return NULL; } fseek(fp, 0, SEEK_END); *pSize = (size_t)(ftell(fp)); if (*pSize == 0) { fclose(fp); return NULL; } result = (unsigned char *)malloc(*pSize); if (result == NULL) { return NULL; } fseek(fp, 0, SEEK_SET); read = fread(result, 1, *pSize, fp); fclose(fp); if (read != *pSize) { free(result); return NULL; } return result; } void LoadFromMemory(void) { void *data; size_t size; HMEMORYMODULE handle; addNumberProc addNumber; HMEMORYRSRC resourceInfo; DWORD resourceSize; LPVOID resourceData; TCHAR buffer[100]; data = ReadLibrary(&size); if (data == NULL) { return; } handle = MemoryLoadLibrary(data, size); if (handle == NULL) { _tprintf(_T("%i"), GetLastError()); _tprintf(_T("Can't load library from memory.\n")); goto exit; } addNumber = (addNumberProc)MemoryGetProcAddress(handle, "addNumbers"); _tprintf(_T("From memory: %d\n"), addNumber(66, 99)); resourceInfo = MemoryFindResource(handle, MAKEINTRESOURCE(VS_VERSION_INFO), RT_VERSION); _tprintf(_T("MemoryFindResource returned 0x%p\n"), resourceInfo); resourceSize = MemorySizeofResource(handle, resourceInfo); resourceData = MemoryLoadResource(handle, resourceInfo); _tprintf(_T("Memory resource data: %ld bytes at 0x%p\n"), resourceSize, resourceData); MemoryLoadString(handle, 1, buffer, sizeof(buffer)); _tprintf(_T("String1: %s\n"), buffer); MemoryLoadString(handle, 20, buffer, sizeof(buffer)); _tprintf(_T("String2: %s\n"), buffer); MemoryFreeLibrary(handle); exit: free(data); } ```

This time, the test program failed, with an access violation.

III. Attempt to use C#'s P/Invoke:

It just works, using a basic DllImport attribute.

IV. Attempt to use a translated MemoryModule version, from a managed executable:

Fails, for the same reason.

V. Some more stuff.

After using a C# debugger on the translated version, and various printf debugging on the original library, I found this line of code's the culprit:

            BOOL successfull = (*DllEntry)((HINSTANCE)code, DLL_PROCESS_ATTACH, 0);

The access violation happened right after calling the dll entry point, and not because of a null pointer.

TL;DR

I do not have any experience with such low level programming, please help me explain why MemoryModule crashed when calling this entry point. Thanks in advance.

trungnt2910 commented 2 years ago

It magically loaded when using --static-libgcc and --static-libstdc++, but the question remains: Why doesn't your code throw a module not found exception, but crash mysteriously?