search

falcosecurity / falco

Cloud Native Runtime Security
https://falco.org
Apache License 2.0
7.39k stars 902 forks source link

Failing Falco-0.24.0 kernel probe for 5.4.50 kernel - bottlerocket os #1411

Closed faarshad closed 4 years ago

faarshad commented 4 years ago

Bug Description I am trying to compile the driver for falco-0.24.0 for 5.4.50 kernel and it is giving me the following error:

Setup: Building the driver from inside of Amazon Linux 2 container image running on top of Bottlerocket OS 1.0.1(aws-k8s-1.16) gcc/g++ version= 7.3.1-9 cmake version = 3.10.0

Errors during compilation:

/usr/local/bin/cmake -H/tmp/falco -B/tmp/falco/build --check-build-system CMakeFiles/Makefile.cmake 0
make -f CMakeFiles/Makefile2 driver
make[1]: Entering directory `/tmp/falco/build'
/usr/local/bin/cmake -H/tmp/falco -B/tmp/falco/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/bin/cmake -E cmake_progress_start /tmp/falco/build/CMakeFiles 0
make -f CMakeFiles/Makefile2 driver/CMakeFiles/driver.dir/all
make[2]: Entering directory `/tmp/falco/build'
make -f driver/CMakeFiles/driver.dir/build.make driver/CMakeFiles/driver.dir/depend
make[3]: Entering directory `/tmp/falco/build'
cd /tmp/falco/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /tmp/falco /tmp/falco/build/sysdig-repo/sysdig-prefix/src/sysdig/driver /tmp/falco/build /tmp/falco/build/driver /tmp/falco/build/driver/CMakeFiles/driver.dir/DependInfo.cmake --color=
make[3]: Leaving directory `/tmp/falco/build'
make -f driver/CMakeFiles/driver.dir/build.make driver/CMakeFiles/driver.dir/build
make[3]: Entering directory `/tmp/falco/build'
cd /tmp/falco/build/driver/src && make
make[4]: Entering directory `/tmp/falco/build/driver/src'
make -C /lib/modules/5.4.50/build M=/tmp/falco/build/driver/src modules
make[5]: Entering directory `/usr/src/kernels/5.4.50'
  CC [M]  /tmp/falco/build/driver/src/main.o
In file included from <command-line>:0:0:
./arch/x86/include/asm/segment.h: In function 'vdso_read_cpunode':
././include/linux/compiler_types.h:210:24: error: expected '(' before '__inline'
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:240:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
  ^~~~~~~~~~
./arch/x86/include/asm/segment.h:266:2: note: in expansion of macro 'alternative_io'
  alternative_io ("lsl %[seg],%[p]",
  ^~~~~~~~~~~~~~
In file included from ./arch/x86/include/asm/barrier.h:5:0,
                 from ./include/linux/compiler.h:256,
                 from ./arch/x86/include/asm/current.h:5,
                 from ./include/linux/sched.h:12,
                 from ./arch/x86/include/asm/syscall.h:14,
                 from /tmp/falco/build/driver/src/main.c:19:
./arch/x86/include/asm/alternative.h:108:2: error: expected identifier or '(' before string constant
  "# ALT: oldnstr\n"      \
  ^
./arch/x86/include/asm/alternative.h:159:2: note: in expansion of macro 'OLDINSTR'
  OLDINSTR(oldinstr, 1)      \
  ^~~~~~~~
./arch/x86/include/asm/alternative.h:240:23: note: in expansion of macro 'ALTERNATIVE'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
                       ^~~~~~~~~~~
./arch/x86/include/asm/segment.h:266:2: note: in expansion of macro 'alternative_io'
  alternative_io ("lsl %[seg],%[p]",
  ^~~~~~~~~~~~~~
In file included from <command-line>:0:0:
././include/linux/compiler_types.h:210:24: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:240:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
  ^~~~~~~~~~
./arch/x86/include/asm/segment.h:266:2: note: in expansion of macro 'alternative_io'
  alternative_io ("lsl %[seg],%[p]",
  ^~~~~~~~~~~~~~
In file included from <command-line>:0:0:
./arch/x86/include/asm/page_64.h: In function 'clear_page':
././include/linux/compiler_types.h:210:24: error: expected '(' before '__inline'
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:256:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE_2("call %P[old]", "call %P[new1]", feature1,\
  ^~~~~~~~~~
./arch/x86/include/asm/page_64.h:49:2: note: in expansion of macro 'alternative_call_2'
  alternative_call_2(clear_page_orig,
  ^~~~~~~~~~~~~~~~~~
In file included from ./arch/x86/include/asm/barrier.h:5:0,
                 from ./include/linux/compiler.h:256,
                 from ./arch/x86/include/asm/current.h:5,
                 from ./include/linux/sched.h:12,
                 from ./arch/x86/include/asm/syscall.h:14,
                 from /tmp/falco/build/driver/src/main.c:19:
./arch/x86/include/asm/alternative.h:128:2: error: expected identifier or '(' before string constant
  "# ALT: oldinstr2\n"         \
  ^
./arch/x86/include/asm/alternative.h:168:2: note: in expansion of macro 'OLDINSTR_2'
  OLDINSTR_2(oldinstr, 1, 2)     \
  ^~~~~~~~~~
./arch/x86/include/asm/alternative.h:256:23: note: in expansion of macro 'ALTERNATIVE_2'
  asm_inline volatile (ALTERNATIVE_2("call %P[old]", "call %P[new1]", feature1,\
                       ^~~~~~~~~~~~~
./arch/x86/include/asm/page_64.h:49:2: note: in expansion of macro 'alternative_call_2'
  alternative_call_2(clear_page_orig,
  ^~~~~~~~~~~~~~~~~~
In file included from <command-line>:0:0:
././include/linux/compiler_types.h:210:24: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:256:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE_2("call %P[old]", "call %P[new1]", feature1,\
  ^~~~~~~~~~
./arch/x86/include/asm/page_64.h:49:2: note: in expansion of macro 'alternative_call_2'
  alternative_call_2(clear_page_orig,
  ^~~~~~~~~~~~~~~~~~
./arch/x86/include/asm/special_insns.h: In function 'clflushopt':
././include/linux/compiler_types.h:210:24: error: expected '(' before '__inline'
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:240:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
  ^~~~~~~~~~
./arch/x86/include/asm/special_insns.h:205:2: note: in expansion of macro 'alternative_io'
  alternative_io(".byte " __stringify(NOP_DS_PREFIX) "; clflush %P0",
  ^~~~~~~~~~~~~~
In file included from ./arch/x86/include/asm/barrier.h:5:0,
                 from ./include/linux/compiler.h:256,
                 from ./arch/x86/include/asm/current.h:5,
                 from ./include/linux/sched.h:12,
                 from ./arch/x86/include/asm/syscall.h:14,
                 from /tmp/falco/build/driver/src/main.c:19:
./arch/x86/include/asm/alternative.h:108:2: error: expected identifier or '(' before string constant
  "# ALT: oldnstr\n"      \
  ^
./arch/x86/include/asm/alternative.h:159:2: note: in expansion of macro 'OLDINSTR'
  OLDINSTR(oldinstr, 1)      \
  ^~~~~~~~
./arch/x86/include/asm/alternative.h:240:23: note: in expansion of macro 'ALTERNATIVE'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
                       ^~~~~~~~~~~
./arch/x86/include/asm/special_insns.h:205:2: note: in expansion of macro 'alternative_io'
  alternative_io(".byte " __stringify(NOP_DS_PREFIX) "; clflush %P0",
  ^~~~~~~~~~~~~~
In file included from <command-line>:0:0:
././include/linux/compiler_types.h:210:24: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:240:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
  ^~~~~~~~~~
./arch/x86/include/asm/special_insns.h:205:2: note: in expansion of macro 'alternative_io'
  alternative_io(".byte " __stringify(NOP_DS_PREFIX) "; clflush %P0",
  ^~~~~~~~~~~~~~
./arch/x86/include/asm/processor.h: In function 'prefetch':
././include/linux/compiler_types.h:210:24: error: expected '(' before '__inline'
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:221:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
  ^~~~~~~~~~
./arch/x86/include/asm/processor.h:795:2: note: in expansion of macro 'alternative_input'
  alternative_input(BASE_PREFETCH, "prefetchnta %P1",
  ^~~~~~~~~~~~~~~~~
In file included from ./arch/x86/include/asm/barrier.h:5:0,
                 from ./include/linux/compiler.h:256,
                 from ./arch/x86/include/asm/current.h:5,
                 from ./include/linux/sched.h:12,
                 from ./arch/x86/include/asm/syscall.h:14,
                 from /tmp/falco/build/driver/src/main.c:19:
./arch/x86/include/asm/alternative.h:108:2: error: expected identifier or '(' before string constant
  "# ALT: oldnstr\n"      \
  ^
./arch/x86/include/asm/alternative.h:159:2: note: in expansion of macro 'OLDINSTR'
  OLDINSTR(oldinstr, 1)      \
  ^~~~~~~~
./arch/x86/include/asm/alternative.h:221:23: note: in expansion of macro 'ALTERNATIVE'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
                       ^~~~~~~~~~~
./arch/x86/include/asm/processor.h:795:2: note: in expansion of macro 'alternative_input'
  alternative_input(BASE_PREFETCH, "prefetchnta %P1",
  ^~~~~~~~~~~~~~~~~
In file included from <command-line>:0:0:
././include/linux/compiler_types.h:210:24: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:221:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
  ^~~~~~~~~~
./arch/x86/include/asm/processor.h:795:2: note: in expansion of macro 'alternative_input'
  alternative_input(BASE_PREFETCH, "prefetchnta %P1",
  ^~~~~~~~~~~~~~~~~
./arch/x86/include/asm/processor.h: In function 'prefetchw':
././include/linux/compiler_types.h:210:24: error: expected '(' before '__inline'
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:221:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
  ^~~~~~~~~~
./arch/x86/include/asm/processor.h:807:2: note: in expansion of macro 'alternative_input'
  alternative_input(BASE_PREFETCH, "prefetchw %P1",
  ^~~~~~~~~~~~~~~~~
In file included from ./arch/x86/include/asm/barrier.h:5:0,
                 from ./include/linux/compiler.h:256,
                 from ./arch/x86/include/asm/current.h:5,
                 from ./include/linux/sched.h:12,
                 from ./arch/x86/include/asm/syscall.h:14,
                 from /tmp/falco/build/driver/src/main.c:19:
./arch/x86/include/asm/alternative.h:108:2: error: expected identifier or '(' before string constant
  "# ALT: oldnstr\n"      \
  ^
./arch/x86/include/asm/alternative.h:159:2: note: in expansion of macro 'OLDINSTR'
  OLDINSTR(oldinstr, 1)      \
  ^~~~~~~~
./arch/x86/include/asm/alternative.h:221:23: note: in expansion of macro 'ALTERNATIVE'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
                       ^~~~~~~~~~~
./arch/x86/include/asm/processor.h:807:2: note: in expansion of macro 'alternative_input'
  alternative_input(BASE_PREFETCH, "prefetchw %P1",
  ^~~~~~~~~~~~~~~~~
In file included from <command-line>:0:0:
././include/linux/compiler_types.h:210:24: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/alternative.h:221:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
  ^~~~~~~~~~
./arch/x86/include/asm/processor.h:807:2: note: in expansion of macro 'alternative_input'
  alternative_input(BASE_PREFETCH, "prefetchw %P1",
  ^~~~~~~~~~~~~~~~~
./include/linux/thread_info.h: In function 'copy_overflow':
././include/linux/compiler_types.h:210:24: error: expected '(' before '__inline'
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/bug.h:35:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile("1:\t" ins "\n"    \
  ^~~~~~~~~~
./arch/x86/include/asm/bug.h:79:2: note: in expansion of macro '_BUG_FLAGS'
  _BUG_FLAGS(ASM_UD2, BUGFLAG_WARNING|(flags));  \
  ^~~~~~~~~~
./include/asm-generic/bug.h:93:3: note: in expansion of macro '__WARN_FLAGS'
   __WARN_FLAGS(BUGFLAG_NO_CUT_HERE | BUGFLAG_TAINT(taint));\
   ^~~~~~~~~~~~
./include/asm-generic/bug.h:124:3: note: in expansion of macro '__WARN_printf'
   __WARN_printf(TAINT_WARN, format);   \
   ^~~~~~~~~~~~~
./include/linux/thread_info.h:134:2: note: in expansion of macro 'WARN'
  WARN(1, "Buffer overflow detected (%d < %lu)!\n", size, count);
  ^~~~
...
...
In file included from <command-line>:0:0:
././include/linux/compiler_types.h:210:24: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
 #define asm_inline asm __inline
                        ^
./arch/x86/include/asm/bug.h:35:2: note: in expansion of macro 'asm_inline'
  asm_inline volatile("1:\t" ins "\n"    \
  ^~~~~~~~~~
./arch/x86/include/asm/bug.h:79:2: note: in expansion of macro '_BUG_FLAGS'
  _BUG_FLAGS(ASM_UD2, BUGFLAG_WARNING|(flags));  \
  ^~~~~~~~~~
./include/asm-generic/bug.h:98:3: note: in expansion of macro '__WARN_FLAGS'
   __WARN_FLAGS(BUGFLAG_ONCE |   \
   ^~~~~~~~~~~~
./include/net/request_sock.h:119:2: note: in expansion of macro 'WARN_ON_ONCE'
  WARN_ON_ONCE(refcount_read(&req->rsk_refcnt) != 0);
  ^~~~~~~~~~~~
make[7]: *** [/tmp/falco/build/driver/src/main.o] Error 1
make[6]: *** [/tmp/falco/build/driver/src] Error 2
make[5]: *** [sub-make] Error 2
make[5]: Leaving directory `/usr/src/kernels/5.4.50'
make[4]: *** [all] Error 2
make[4]: Leaving directory `/tmp/falco/build/driver/src'
make[3]: *** [driver/CMakeFiles/driver] Error 2
make[3]: Leaving directory `/tmp/falco/build'
make[2]: *** [driver/CMakeFiles/driver.dir/all] Error 2
make[2]: Leaving directory `/tmp/falco/build'
make[1]: *** [driver/CMakeFiles/driver.dir/rule] Error 2
make[1]: Leaving directory `/tmp/falco/build'
make: *** [driver] Error 2

How to reproduce it On bottlerocket-os, login into the admin container which is a amazon-linux-2 container & execute the following for installing the build toolchain and then building the falco probe:

#Build tool chain
yum install gcc gcc-c++ git make autoconf automake pkg-config patch ncurses-devel libtool glibc-static libstdc++-static elfutils-libelf-devel libcurl libcurl-devel wget openssl-devel which -y

cd /tmp/ && wget https://cmake.org/files/v3.10/cmake-3.10.0.tar.gz && tar zxvf cmake-3.10.0.tar.gz && cd cmake-3.10.0 && ./bootstrap --system-curl && make && make install

#Build falco:
cd /tmp && git clone https://github.com/falcosecurity/falco.git ;cd falco; git checkout 0.24.0; mkdir -p build;cd build;cmake -DUSE_BUNDLED_DEPS=ON -DCMAKE_VERBOSE_MAKEFILE=On ..

make driver

Expected behaviour A kernel module built in build/driver/src/falco.ko

Environment

Additional context

This issue seems similar to https://github.com/falcosecurity/falco/issues/1405 Also, Missing support for asm_inline in Linux 5.4 could be the issue.

faarshad commented 4 years ago

I was able to build the falco probe by installing gcc-9.2.0 from source on amazon linux 2 admin container of bottlerocket. I followed the procedure listed here to install gcc-9.2.0.

Following logs show that falco driver was built:

bash-4.2# make driver
/usr/local/bin/cmake -H/tmp/falco -B/tmp/falco/build --check-build-system CMakeFiles/Makefile.cmake 0
make -f CMakeFiles/Makefile2 driver
make[1]: Entering directory `/tmp/falco/build'
/usr/local/bin/cmake -H/tmp/falco -B/tmp/falco/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/bin/cmake -E cmake_progress_start /tmp/falco/build/CMakeFiles 0
make -f CMakeFiles/Makefile2 driver/CMakeFiles/driver.dir/all
make[2]: Entering directory `/tmp/falco/build'
make -f driver/CMakeFiles/driver.dir/build.make driver/CMakeFiles/driver.dir/depend
make[3]: Entering directory `/tmp/falco/build'
cd /tmp/falco/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /tmp/falco /tmp/falco/build/sysdig-repo/sysdig-prefix/src/sysdig/driver /tmp/falco/build /tmp/falco/build/driver
/tmp/falco/build/driver/CMakeFiles/driver.dir/DependInfo.cmake --color=
Scanning dependencies of target driver
make[3]: Leaving directory `/tmp/falco/build'
make -f driver/CMakeFiles/driver.dir/build.make driver/CMakeFiles/driver.dir/build
make[3]: Entering directory `/tmp/falco/build'
cd /tmp/falco/build/driver/src && make
make[4]: Entering directory `/tmp/falco/build/driver/src'
make -C /lib/modules/5.4.50/build M=/tmp/falco/build/driver/src modules
make[5]: Entering directory `/usr/src/kernels/5.4.50'
  CC [M]  /tmp/falco/build/driver/src/main.o
  CC [M]  /tmp/falco/build/driver/src/dynamic_params_table.o
  CC [M]  /tmp/falco/build/driver/src/fillers_table.o
  CC [M]  /tmp/falco/build/driver/src/flags_table.o
  CC [M]  /tmp/falco/build/driver/src/ppm_events.o
  CC [M]  /tmp/falco/build/driver/src/ppm_fillers.o
  CC [M]  /tmp/falco/build/driver/src/event_table.o
  CC [M]  /tmp/falco/build/driver/src/syscall_table.o
  CC [M]  /tmp/falco/build/driver/src/ppm_cputime.o
  LD [M]  /tmp/falco/build/driver/src/falco.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC [M]  /tmp/falco/build/driver/src/falco.mod.o
  LD [M]  /tmp/falco/build/driver/src/falco.ko
make[5]: Leaving directory `/usr/src/kernels/5.4.50'
make[4]: Leaving directory `/tmp/falco/build/driver/src'
cd /tmp/falco/build/driver/src && /usr/local/bin/cmake -E copy_if_different falco.ko /tmp/falco/build/driver
make[3]: Leaving directory `/tmp/falco/build'
Built target driver
make[2]: Leaving directory `/tmp/falco/build'
/usr/local/bin/cmake -E cmake_progress_start /tmp/falco/build/CMakeFiles 0
make[1]: Leaving directory `/tmp/falco/build'

Load the driver in bottlerocket-os by using sudo sheltie and insmod from admin container

sudo sheltie
insmod /run/host-containerd/io.containerd.runtime.v2.task/default/admin/rootfs/tmp/falco/build/driver/src/falco.ko

It might take some time to get the module loaded into the kernel. Verify my running lsmod

bash-5.0# lsmod | more
Module                  Size  Used by
falco                 638976  2
axot commented 3 years ago

I think this issue keep happening, is there a plan to fix this?