Zeex / subhook

Simple hooking library for C/C++ (x86 only, 32/64-bit, no dependencies)
BSD 2-Clause "Simplified" License
798 stars 124 forks source link

Fail to run in MacOS Catalina 10.15.4 #45

Closed tobegit3hub closed 4 years ago

tobegit3hub commented 4 years ago

Here is the error log.

$ make test
Running tests...
Test project /Users/tobe/code/subhook/buid
    Start 1: subhook_test_exe_test
1/2 Test #1: subhook_test_exe_test ............***Failed  Required regular expression not found. Regex=[Testing initial install
foo_hooked\(\) called
foo\(\) called
Testing re-install
foo_hooked\(\) called
foo\(\) called
Testing trampoline
foo_hooked_tr\(\) called
foo\(\) called

]  0.00 sec
    Start 2: subhook_cxx_test_exe_test
2/2 Test #2: subhook_cxx_test_exe_test ........***Failed  Required regular expression not found. Regex=[Testing initial install
foo_hooked\(\) called
foo\(\) called
Testing re-install
foo_hooked\(\) called
foo\(\) called
Testing trampoline
foo_hooked_tr\(\) called
foo\(\) called

]  0.00 sec

0% tests passed, 2 tests failed out of 2

Total Test time (real) =   0.01 sec

The following tests FAILED:
          1 - subhook_test_exe_test (Failed)
          2 - subhook_cxx_test_exe_test (Failed)
Errors while running CTest
make: *** [test] Error 8
tobegit3hub commented 4 years ago

It seems that subhook_install fail with MacOS Catalina 10.15.4 and the tests will work in Linux.

tobegit3hub commented 4 years ago

We have traced the code and find out the system call of mprotect was failed in new MacOS.

The following function returns zero in MacOS and make the _hook object as NULL.

if (mprotect(address, size, PROT_READ | PROT_WRITE | PROT_EXEC) == 0) ...

Same issue for https://github.com/agiledragon/gomonkey/issues/10 .

Zeex commented 4 years ago

https://stackoverflow.com/questions/60654834/using-mprotect-to-make-text-segment-writable-on-macos

As mentioned in this post, in Catalina the linker sets maxprot in 64-bit executables to read+execute rather than read+write+execute as in earliner versions, which makes mprotect fail if PROT_WRITE is passed in the protection flags parameter.

It seems that the only way to make hooking work there is to manually overwrite the flags after compiling your program.

For the subhhok's test program I used this command from the SO post:

printf '\x07' | dd of=tests/Debug/test bs=1 seek=160 count=1 conv=notrunc
tobegit3hub commented 4 years ago

Great and this could fix for me.

printf '\x07' | dd of=tests/test bs=1 seek=160 count=1 conv=notrunc
printf '\x07' | dd of=tests/test++ bs=1 seek=160 count=1 conv=notrunc