Hpmason / retour-rs

A cross-platform detour library written in Rust
Other
99 stars 18 forks source link

Crash on enabled detour #48

Closed tsuza closed 7 months ago

tsuza commented 7 months ago

Hello. I've been trying to detour Half-Life 1's LoadLibraryA, though whenever I load it, the program entirely crashes. The hooks is properly initialized. It looks like it crashes right before loadlibrarya_detour gets called ( or at least shortly after it gets enabled ). I attached a debugger to it to see what's the cause, and it's something caused by the code below. I'm unsure whether I'm doing something wrong or what. I also tried copying-and-pasting the example provided in the repository, but it crashes regardless.

The game is 32bit. I'm compiling my dll as i686-pc-windows-msvc.

type FnLoadLibraryA = unsafe extern "system" fn(PCSTR) -> HMODULE;

static_detour! {
  static LoadLibraryAHook: unsafe extern "system" fn(PCSTR) -> HMODULE;
}

#[ctor]
fn main_dll() {
    let load_library_address =
        get_module_symbol_address("kernel32.dll\0", "LoadLibraryA\0").unwrap();

    let load_library_function: FnLoadLibraryA =
        unsafe { std::mem::transmute(load_library_address) };

    unsafe {
        LoadLibraryAHook
            .initialize(load_library_function, loadlibrarya_detour)
            .unwrap()
            .enable()
            .unwrap()
    };
}

fn get_module_symbol_address(module: &str, symbol: &str) -> Option<usize> {
    unsafe {
        let handle = GetModuleHandleA(PCSTR(module.as_ptr())).unwrap();
        match GetProcAddress(handle, PCSTR(symbol.as_ptr() as _)) {
            Some(func) => Some(func as usize),
            None => None,
        }
    }
}

fn loadlibrarya_detour(lpFileName: PCSTR) -> HMODULE {
    unsafe { LoadLibraryAHook.disable().unwrap() };
    let ret = unsafe { LoadLibraryAHook.call(lpFileName) };
    unsafe { LoadLibraryAHook.enable().unwrap() };

    ret
}
tsuza commented 7 months ago

Alright, after wasting hours upon hours, I've found out that it was the asi loader somehow causing the crash (?????????). Loading the DLL through other means makes the detour work just fine. I'll be closing this issue. Sorry for opening an issue that ended up being caused by something else.

Hpmason commented 7 months ago

All good, glad you were able to find your issue!