Standardize on dynamic library symbols published by libsystrace

[rrnewton]

@wangbj - I'm not a big fan of combining the raw/traced functionality into one function. How about the below two functions as the symbols defined by the libsystrace.so library? (Yes, I understand that right now there is no libsystrace.so and we have the additional stipulation that the user must launch via the bpf-trace binary.)

// A dummy version is provided by libsystrace.so, over-ridden by subsequently loaded libs:
int captured_syscall(int syscallno, long a0, long a1, long a2, long a3, long a4, long a5);

// This is the "backdoor" provided by libsystrace.so:
int raw_syscall(int syscallno, long a0, long a1, long a2, long a3, long a4, long a5);

Here's the justification re: the above naming proposal:

• "traced" implies a particular use case to me -- that you're observing the functions going by (like systrace), rather than intercepting them and replacing them. If captured_syscall does NOT execute the semantics of the captured syscall then it never happened!

The trivial implementation of captured_syscall should just be to call the raw one:

int captured_syscall(int syscallno, long a0, long a1, long a2, long a3, long a4, long a5) {
    raw_syscall(syscallno, a0, a1, a2, a3, a4, a5);
}

I propose that that's the dummy one provided by libsystrace.so. Or maybe it could even print out a message by default (like strace), so you know that you failed to put a tool on top, or error and say that it needs you to load another lib with the instrumentation tool... I'm agnostic on that.

Also, @wangbj indicated that there should be some minimal set of other raw_ wrappers for convenience (e.g. printing to stdout). What should the initial set of those be?

Usage example

Eventually, load libsystrace to hook things, and libdet to override captured_syscall:

LD_PRELOAD="libsystrace.so libdet.so" myprog args..

Currently:

LD_PRELOAD="libdet.so" bpf-trace myprog args..

[wangbj]

comitted 99413fbe8bfb358dc61b8ad40934266d8c619c0d, so there're three APIs are exposed:

• captured_syscall, where we dispatch our intercepted syscalls, defined as a weak function, by default it calls untraced_syscall to allow the syscall going through;

• traced_syscall: initiate a syscall, which will be traced by seccomp-bpf, might ends up into captured_syscall

• untraced_syscall: initiate a syscall, but will not be traced by seccomp-bpf.

[wangbj]

For libdet.so, it can provide a captured_syscall function, to override the weak one mentioned above.

[rrnewton]

@wangbj - how about the other untraced_ (orraw_) calls? Would raw fputs or fprintf be candidates for making our internal libdet code easier to write?

Or ... can we just use seccomp bpf to whitelist our whole libdet library and somehow use the Rust Core thing, statically linked, to ensure that all syscalls within "libdet" are go through untrapped?

[wangbj]

I couldn't find a good way to tell rustc link with musl libc.a, which is very easy to do with gcc/clang.

there're two options seems feasible:

1) use rust core (no_std) for libdet.so; it is not as easy as libstd, and we need to do some work such as connecting the pieces; 2) build a modified musl libc.a, the modified version will use -fvisibility-hidden to hide most symbols to avoid conflicts with glibc, because of LD_PRELOAD; we can also force SYSCALL_NO_INLINE to force all syscalls in musl libc (libc.a) use __syscall function, which we can override in one central piece. The downside is it couldn't find a way to tell rust/cargo to link our modified libc.a, so might ends up using C instead of rust.

[rrnewton]

@wangbj - should we also expose init (or shutdown) callbacks from the client of libsystrace, i.e.libdet.so?

Probably it is sufficient for libdet.so to contain init code (__attribute__((constructor))). The only offhand reason I can think for having an libsystem call a libdet init function would be for it to return some description of what syscalls it wants to "subscribe to" with captured_syscall callbacks.

[wangbj]

rust doesn't have a good way to use attribute((constructor)) or attribute((destructor)), OTOH libsystrace.so does have them as it was written in pure C. libdet.so can have some init function, can you give a prototype of the init function you were thinking?

[gatoWololo]

rust doesn't have a good way to use attribute((constructor)) or attribute((destructor)),

What about something like this?

[wangbj]

learnt something new :)