pgoodman
commented
10 years ago The issue here is really: the system call functions are called indirectly via some entrypoint code targeted by the syscall-specific MSR. This code sets up the gs segment registers, switches stacks, saves/restores registers, and sometimes does sanity checks. Summary: it is very low-level code, some of which--if instrumented--might not work with Granary (particularly the use of swapgs).
If transparent returns are enabled, then we'll start by instrumenting the syscall entrypoints with RetAddrInCodeCache::returns_to_cache == true, but once we call another function, this will become false, and so when that function eventually returns, it will be false, and when the system function itself returns, that return will be instrumented, and translation will continue into the code that eventually leads to swapgs and sysret et al. This will be bad!
This issues occurs when transparent returns are enabled for kernel mode instrumentation. The kernel entrypoints are the system call functions themselves (as opposed to the real system call entrypoint). These functions, when instrumented, should go native when they return. It's not clear what the right way to accomplish this is. One approach is to use an inverted index of code cache block heads to meta-data. This will be useful and needed anyway, so is potentially the ideal case. Another approach is to wrap the entrypoints with an extra call, as is done with function wrappers. This seems less ideal.