syscalls blacklist

[milabs]

It's impossible to hook several system calls by just replacing correspinding sys_call_table values. So, we need to blacklist that numbers which can be found by the command:

cat /proc/kallsyms | grep -e 'T stub'

My x86_64 system shows the result:

0000000000000000 T stub_clone
0000000000000000 T stub_fork
0000000000000000 T stub_vfork
0000000000000000 T stub_sigaltstack
0000000000000000 T stub_iopl
0000000000000000 T stub_execve
0000000000000000 T stub_rt_sigreturn
0000000000000000 T stub32_rt_sigreturn
0000000000000000 T stub32_sigreturn
0000000000000000 T stub32_sigaltstack
0000000000000000 T stub32_execve
0000000000000000 T stub32_fork
0000000000000000 T stub32_clone
0000000000000000 T stub32_vfork
0000000000000000 T stub32_iopl

1. What's about x86_32?
2. Do we needed to find the proper way to handling these are syscalls?

[alexandernst]

Hmmm. Not sure about those one. Some of them doesn't sound familiar, while others are must-have in Procmon (fork and execve, e.g.). Currently I'm downloading a few Ubuntu images (with old kernels) to test procmon on them. I'll attach the output of that command when I have some of the VBox running :)

[milabs]

@alexandernst IIRC VirtualBox had problems with sidt instruction emulation...

[alexandernst]

@milabs Hi! I was looking at this bug. How are those syscalls replaced? Why are they different? How do they work? And most importantly, how do I hijack them?

[milabs]

@alexandernst See how the stub_\func macro implemented: http://lxr.free-electrons.com/source/arch/x86/kernel/entry_64.S?v=3.10#L832

I'll suggest that we can: 1) copy each affected stub function from the kernel's memory OR write the stub's implementation by hands 2) fixup the call to sys_... function (call sys_\func in the macro).

What do you think, master?

[alexandernst]

Hmmm... not really sure, I feel kind of lost in this issue, but maybe it would be better if we patch the sys_... call, this way even if future versions of the kernel change how stubs work, it will keep working (because it will, right?).

But then... if we do that, we'd need to call the original stub from inside our stub, which is not how other syscalls are hooked (note that in current implementation, fake syscalls are called after the original syscall).

How would you do it? Or said with other words, what's your best choise and what pros/cons does it has?

[alexandernst]

I'm pasting here a fragment of a talk I had with another guy about his a few weeks (months?) ago.

<alexandernst> LePhilousophe: remember I once told you that hijacking __NR_read is easy because it's just a function address, but hijacking stub_fork is different, even I didn't know <how> it's different, and you affirmed. So... I got to the point where I'm actually interested in knowing <how> is different and how can I hijack it. Can you explain me a little bit about it or tell me where could I read about that, please? :)
<LePhilousophe> what ? :)
<LePhilousophe> I don't even understand what you are telling :(
<LePhilousophe> ok __NR_READ is the syscall index
<LePhilousophe> I don't see where stub_fork is defined
<alexandernst> ah... maybe it wasn't you? :/
<alexandernst> thing is, some (most) syscalls are plain functions, but there are a few (around 10) that are different
<alexandernst> __NR_clone, fork, vfork, execve and some other
<alexandernst> cat /proc/kallsyms | grep -e 'T stub'     <------- the output of this
<LePhilousophe> hum
<LePhilousophe> but what do you hijack ?
<LePhilousophe> syscalls or more ?
<alexandernst> only syscalls
<alexandernst> but __NR_fork is a syscall
<LePhilousophe> so what fails ?
<alexandernst> it's just that it works in a different way 
<LePhilousophe> btw never thought about it before
<alexandernst> the syscall table is a simple array that points (mostly) to functions. Replacing the address of a syscall is all I need to hijack a syscall, except the ones I mentioned before
<LePhilousophe> but why don't you just hijack interrupt ?
<alexandernst> interrupt?
<alexandernst> ah, 0x80 you mean?
<LePhilousophe> you know how syscalls are called ?
<LePhilousophe> yeah
<Ford_Prefect> Isn't there a syscall instruction these days?
<alexandernst> I guess it would be possible, yeah
<LePhilousophe> Ford_Prefect: there is
<alexandernst> Ford_Prefect: yes for x64
<LePhilousophe> but you can hijack it too
<alexandernst> hmmm
<alexandernst> ok, I'll think about it
<LePhilousophe> alexandernst: anyway that doesn't change your question
<LePhilousophe> I don't know... yet
<alexandernst> LePhilousophe: yeah, those stub-syscalls are weird
<LePhilousophe> alexandernst: http://lxr.free-electrons.com/source/arch/x86/kernel/entry_64.S#L862
<alexandernst> LePhilousophe: yes, there is where it's created (or executed?)
<LePhilousophe> so it's still a function pointer
<LePhilousophe> I don't understand what is wrong
<alexandernst> I was told it won't work just by replacing the address in the syscall table :/
<alexandernst> but the guy who told me this (on #kernel) had an extremely bad english and I couldn't understand what exactly am I supposed to do to hijack those stubs
<LePhilousophe> did you try ?
<LePhilousophe> http://lxr.free-electrons.com/source/arch/x86/kernel/entry_64.S#L593
<LePhilousophe> I don't understand what would prevent you to do it
<alexandernst> actually I didn't try :(
<LePhilousophe> am I right if I say that you catch syscall before execution and not after ?
<LePhilousophe> and that you never try to catch it after
<alexandernst> no to the first part, yes to the second part
<alexandernst> I don't catch "syscall" (which is the dispatcher)
<LePhilousophe> hum no I mistold it
<alexandernst> I catch different syscalls
<LePhilousophe> yeah by replacing the pointer in the syscall table
<alexandernst> but I do catch them before they are executed
<alexandernst> exactly
<LePhilousophe> but after you did that how do you call original syscall ?
<alexandernst> here comes the tricky part :)
<alexandernst> I create a stub (an executable memory that I create for each syscall) and I fill that stub with opcode. The opcode looks like this basically:
<alexandernst> 1. call real syscall addr. 2. save result in stack. 3. call fake syscall with the same arguments. 
<LePhilousophe> ok
<LePhilousophe> so he was right
<LePhilousophe> I will try to explain
<alexandernst> :)
<LePhilousophe> if you look at all stub things
<LePhilousophe> you will notice they all end with jmp int_ret_from_sys_call
<LePhilousophe> so they will never return to your code
<alexandernst> aaahhhh
<LePhilousophe> because what you do is call and not jmp
<LePhilousophe> but you have to if you want to get the return value
<alexandernst> so, as how I see it, one possible solution would be to clone each stub and patch the jmp instruction, right?
<alexandernst> (and then replace the addr of the real stub with the fake one)
<LePhilousophe> hum
<LePhilousophe> that could work but it's dangerous :D
<alexandernst> :D 
<alexandernst> I love dangerous
<LePhilousophe> I mean if in some version of the kernel the stub changes you won't have the same changes
<alexandernst> ouch
<LePhilousophe> that could break everything and you can say goodbye to profit
<alexandernst> my moneiz?!?! :(
<v> You could replace the sysret (or whatever it is) in the real with a jump to your code, and finish your code with whatever was there (ie, sysret).
<v> Assuming it's not a relative jump...
<LePhilousophe> moreover, if you look carefully to these stubs you can see a common point
<LePhilousophe> they are all related to creating a new process
<LePhilousophe> that's why they have a different code path than other syscalls
<alexandernst> yup, I noticed that. Thought it's because of some security measures
<alexandernst> hmmm, ok, I'll think about how to hijack those without breaking everything
<alexandernst> thank you LePhilousophe :)  You too v!
<LePhilousophe> alexandernst: in fact the int_ret_from_sys_call doesn't return to userspace code which called the syscall
<LePhilousophe> or not in all paths
<alexandernst> (!)
<v> Well, the execve family only returns on error.
<alexandernst> so the stubs (which are already different from the normal syscalls) are also different between the?!
<v> Could that be it ?
<alexandernst> s/the/them
<alexandernst> I clearly should write a mail to Linus telling him to fix all this non-sense!
<alexandernst> Subject: Fix that stub thingy!
<LePhilousophe> alexandernst: in fact I may have generalized abusively
<LePhilousophe> fork doens't seem to jump to int_ret_from_sys_call
<alexandernst> Message: I'm doing profitz here and your stubs are just in the middle! Fix them and I'll share my moneiz with you! 
<LePhilousophe> http://lxr.free-electrons.com/source/arch/x86/kernel/entry_64.S#L816
<alexandernst> LePhilousophe: :(  so they are different in every single aspect
<LePhilousophe> I think you can monitor fork vfork clone
<alexandernst> they do have one common thing
<alexandernst> they all have a "call sys_\func"
<LePhilousophe> alexandernst: yeah but stub_\func doesn't do anything really strange
<alexandernst> if I could hack somehow the stack, and make the hijacked stubs think they are real, and make them read the stack... and call the real func...
<LePhilousophe> stub_iopl seems to be nice too
<LePhilousophe> the badass are execve and sigreturn
<LePhilousophe> because execve doesn't return if success
<LePhilousophe> sigreturn I don't know why
<LePhilousophe> comment above the stub give some explaination

[alexandernst]

@milabs Hi! Long time without activity here :) How is life going? I got a mail from a guy who actually hooked execve with monks without patching anything: http://es.scribd.com/doc/212269877/execve

I'm not really sure how or why is this working. In fact, I have no idea at all! Do you? :)

[milabs]

@alexandernst He used jprobes as that nice commit says 克隆别人的代码，稍作修改，加入jprobe :) We don't want it I think.

[alexandernst]

@milabs Ah, indeed! I completely missed it. Nope, we don't want that, hehehe. Let's see if I can get some free time in the next few weeks so I can finish this bug and release 0.1 :) I have been wanting to do if for 2 months so far!

[milabs]

@alexandernst good luck ;)

[igorastds]

@milabs I think that sys_execve intercept is working @ 64 bit kernel.. at least I got it working on 3.18. Need details?

[alexandernst]

@igorastds Hmm, that's weird... It shouldn't work. Can you pastebin the function you used to hijack sys_execve?

[milabs]

@alexandernst @igorastds Well, need to remember that things... )

[alexandernst]

Link of interest: https://github.com/kfiros/execmon