Proof of concept checklist

[rrnewton]

@wangbj has already taken this prototype pretty far. Here's a checklist and what's left (feel free to edit/correct):

• [x] trap syscalls in the original binary (bpf)
• [x] in the signal handler, patch the offending instruction with a jump
• [x] JIT code in a bpf "whitelisted" zone for the patched JMP to jump to (the trampoline)
• [x] in the trampoline: place the relocated syscall instruction (temporary: this will eventually be the job of the handler function)
• [x] in the trampoline: capture the register file to the stack
• [x] in the trampoline: set up a call to the handler function
• [ ] in the trampoline: add a pointer to the register file as an argument before calling the handler function
• [ ] in the trampoline: add global state / process state pointers to the handler function also

[rrnewton]

@gatoWololo - this is a place we need to sync up. The low-level JITed code in the trampoline needs to match conventions on the memory layout of the structs pointed to by the arguments to the handler function.

They will have no types in the low-level JIT'd code, but should have a matching type in the Rust code (with a corresponding pragma to control the layout).

[wangbj]

in the trampoline: set up a call to the handler function in the trampoline: add a pointer to the register file as an argument before calling the handler function

This is already possible, you can call register_syscall_hook

https://github.com/iu-parfunc/systrace/blob/c6377541d3b377f8ea92794851fadfe0d609d3c8/src/syscallbuf.c#L92

to register a customized hook. though the caller must be also in the same address space as libpreload.so.

[rrnewton]

Great, check it off! ;-)

[rrnewton]

I think the last two in that original list are debatable:

• We don't need to capture the whole register file if the abstraction is you just get the argument registers.
• The global state issue comes back to symmetry breaking / acquiring PID/TID.

For now, I'm running on the assumption that existing thread local storage (TLS) approaches will be good enough for us to cheaply figure out who we are / where we're running from inside the captured_syscall callback. Perhaps microbenchmark results will prove me wrong on this! I haven't benchmarked TLS in a while, but I did before when releasing a Haskell lib for it.