Unknown symbol in module with kernel 4.18

[StephGbzh]

I'm trying to compile and use this module but insmod fails (no modification of the code, just "make").

sudo insmod ftrace_hook.ko
insmod: ERROR: could not insert module ftrace_hook.ko: Unknown symbol in module

I am using Ubuntu 18.10 with kernel 4.18.0-15 (up-to-date).

dmesg does not print a single line about this and insmod has no verbose mode. I don't see any "export" or "symbol" in the code.

Any idea as to what happens and how to fix this ?

The result of make is not as clean as the one in the README but there are only warnings so it may not be the source of the problem:

make
make -C /lib/modules/4.18.0-15-generic/build M=/home/stephane/Dev/linux-kernel/ftrace-hook modules
make[1]: Entering directory '/usr/src/linux-headers-4.18.0-15-generic'
  CC [M]  /home/stephane/Dev/linux-kernel/ftrace-hook/ftrace_hook.o
/home/stephane/Dev/linux-kernel/ftrace-hook/ftrace_hook.o: warning: objtool: duplicate_filename()+0x1c: call without frame pointer save/setup
/home/stephane/Dev/linux-kernel/ftrace-hook/ftrace_hook.o: warning: objtool: fh_sys_execve()+0x15: call without frame pointer save/setup
/home/stephane/Dev/linux-kernel/ftrace-hook/ftrace_hook.o: warning: objtool: fh_init()+0x12: call without frame pointer save/setup
/home/stephane/Dev/linux-kernel/ftrace-hook/ftrace_hook.o: warning: objtool: fh_exit()+0xc: call without frame pointer save/setup
/home/stephane/Dev/linux-kernel/ftrace-hook/ftrace_hook.o: warning: objtool: fh_sys_clone()+0x24: call without frame pointer save/setup
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/stephane/Dev/linux-kernel/ftrace-hook/ftrace_hook.mod.o
  LD [M]  /home/stephane/Dev/linux-kernel/ftrace-hook/ftrace_hook.ko
make[1]: Leaving directory '/usr/src/linux-headers-4.18.0-15-generic'

[ilammy]

Hmm... I can confirm that I see the same error during insmod with a fresh Ubuntu 18.10 server install (4.18.0-15-generic kernel). I don't see these compilation warnings, but I don't think they are the problem.

Looking for the missing symbols manually does not reveal anything: all symbols are there in the kernel.

for symbol in $(nm ftrace_hook.ko | awk '$1 == "U" { print $2 }')
do
    egrep "\b$symbol\b" /proc/kallsyms || echo "not found: $symbol"
done

So this must be caused by hook initialization error: insmod fails with an error returned by the module initialization routine. Indeed, setting kernel log level to KERN_DEBUG

sudo sh -c 'echo 7 > /proc/sys/kernel/printk'

reveals in dmesg that apparently we can't find the symbol sys_clone when we try to hook it. (But there is __x64_sys_clone instead.) Further investigation has led me to this patch set: "syscalls: clean up stub naming convention" which changed the naming convention for syscalls to what it is now, x86_64 being fucking special again.

Fixing up the names works for me so thankfully there are no ABI changes, or so it seems. I'll cook up a patch soon, let's see if that works for you.

[ilammy]

Also, thanks for finding this out!

[StephGbzh]

Thanks for looking into it and especially describing the steps you took to find the root cause.

That made me realize how little I know about this topic. I did not even know the nm command existed or that you could change the kernel log level like this.

First thing, I changed the code to have this:

static struct ftrace_hook demo_hooks[] = {
    HOOK("__x64_sys_clone",  fh_sys_clone,  &real_sys_clone),
    HOOK("__x64_sys_execve", fh_sys_execve, &real_sys_execve),
};

And that indeed works now !

Still, I have a question about the log level, I did this:

> cat /proc/sys/kernel/printk
4   4   1   7
> sudo sh -c 'echo 7 > /proc/sys/kernel/printk'
> cat /proc/sys/kernel/printk
7   4   1   7

But even with that, loading the faulty module with insmod gave Unknown symbol in module and still nothing in dmesg. Did you change something else to have dmesg say something useful about sys_clone missing ?

[ilammy]

You're welcome to the amazing world of kernel development ;) More fascinating dicoveries are waiting for you ahead.

Regarding the log levels. Error messages about unresolved symbols are printed out using pr_debug. These macros are normally compiled out (unless the whole kernel is compiled for debugging). You can enable them on per-file basis by adding the following line to the Makefile:

CFLAGS_ftrace_hooks := -DDEBUG

Then setting the logging level should make them appear.

[KTalinki]

hi, I am trying to hook sys_open and some other syscalls to intercept the file system activity, on kernel 4.17/4.18 on x86_64.

/proc/kallsyms shows x64_sys_open, but not ksys_open. Not sure if I need to hook ksys_open or x64_sys_open I am getting compilation error of 'undelcared identifier' for sys_open & __x64_sys_open.

What are the right headers I need to include ?

Along with sys_open, I need to hook sys_creat, sys_openat, sys_execve, sys_truncate, sys_ftruncate, sys_write, etc...

Before kernel 4.17, I was able to find the symbol for sys_xyz, but from Kernel 4.17 onward, I am not sure if I need to hook __x64_sys_xyz or ksys_xyz or sys_xyz.

In syscalls.h Some system call are declared with /__ARCH_WANT_SYSCALL_DEPRECATED/

Please help, thank you, Kumar T