microsoft / Detours

Detours is a software package for monitoring and instrumenting API calls on Windows. It is distributed in source code form.
MIT License
5k stars 978 forks source link

Add context callback support for x86 and x64 #257

Closed JohnMcPMS closed 1 year ago

JohnMcPMS commented 1 year ago

Adds a context callback function that is invoked before the detour itself.

Only supports x86 and x64, as that is all I had in the source of the port. Is this acceptable? It should fail when attempted on another architecture in this PR; does it need to work on all supported architectures?

The additional bytes required for every trampoline are likely to be viewed as a problem. Because of that, I made it a conditional compilation feature. The interface remains the same, but if one attempts to pass in a callback without having compiled in support, the detour attach fails.

Other issues that @jaykrell pointed out, but don't seem to be necessary for the shipping use case:

  1. The "trampoline" only preserves the integer fastcall registers, not the floating point registers. Since the shipping use case is to call TlsSetValue, this doesn't end up being an issue.
  2. No pdata/xdata equivalents are created (presumably via RtlAddFunctionTable) for the "trampoline". This doesn't appear to be an issue for the shipping use case, but I agree that it is undesirable for general use.
Microsoft Reviewers: Open in CodeFlow