lsif-cpp and android kernel

[s1341]

Hi,

Following up from https://github.com/sourcegraph/sourcegraph/issues/9, trying to get useful lsif data from the android kernel.

I did the following:

1. I cloned the wahoo-pie kernel build tree from android.googlesource.com, using repo:

   mkdir wahoo-pie
   cd wahoo-pie
   repo init -u https://android.golsoglesource.com/kernel/manifest -b android-msm-wahoo-4.4-pie-qpr2
   repo sync

2. I cloned the latest android clang prebuilts from android.googlesource.com:

   git clone https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86 aosp-clang

3. I patched the build.config file (in the wahoo-pie root) to point it to my aosp-clang toolchain:

   diff --git a/build.config b/build.config
   index 471f62b386f4..2e9c813b1972 100644
   --- a/build.config
   +++ b/build.config
   @@ -8,7 +8,7 @@ DEFCONFIG=wahoo_defconfig
   EXTRA_CMDS=''
   KERNEL_DIR=private/msm-google
   POST_DEFCONFIG_CMDS='check_defconfig'
   -CLANG_PREBUILT_BIN=prebuilts-master/clang/host/linux-x86/clang-4053586/bin/
   +CLANG_PREBUILT_BIN=aosp-clang/clang-r365631c/bin/
   LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN=prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin
   LINUX_GCC_CROSS_COMPILE_ARM32_PREBUILTS_BIN=prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin
   LZ4_PREBUILTS_BIN=prebuilts-master/misc/linux-x86/lz4

4. I patched the kernel root makefile (in private/msm-google) to match CC containing 'clang' instead of exactly equal to 'clang':

   
   diff --git a/Makefile b/Makefile
   index 7d990fa2352a..241764ed7591 100644
   --- a/Makefile
   +++ b/Makefile
   @@ -375,7 +375,7 @@ CFLAGS_GCOV = -fprofile-arcs -ftest-coverage -fno-tree-loop-im
   CFLAGS_KCOV    = -fsanitize-coverage=trace-pc

-ifeq ($(cc-name),clang) +ifeq ($(findstring clang,$(cc-name)),clang) ifneq ($(CROSS_COMPILE),) CLANG_TRIPLE ?= $(CROSS_COMPILE) CLANG_TARGET := --target=$(notdir $(CLANG_TRIPLE:%-=%))

5. I added symlinks to 'clang' to the target-specific gcc toolchain bin directories, to stop clang from using `/usr/bin/as`. (This is according to https://blog.printk.io/2015/03/cross-compile-linux-for-arm-using-llvm-clang-on-arch-linux/):

pushd prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin ln -s ../../../../../../aosp-clang/clang-r365631c/bin/clang.real clangp popd

pushd prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin ln -s ../../../../../../aosp-clang/clang-r365631c/bin/clang.real clang popd

6. Patched `build/build.sh` to wrap the kernel build 'make' with generate-csv:

diff --git a/build.sh b/build.sh index 73eb3fc..f72cc57 100755 --- a/build.sh +++ b/build.sh @@ -268,7 +268,7 @@ echo "========================================================" echo " Building kernel"

set -x -(cd ${OUT_DIR} && make O=${OUT_DIR} "${TOOL_ARGS[@]}" ${MAKE_ARGS}) +(cd ${OUT_DIR} && env CLEAN=true ABSOUTDIR=/tmp/lsifout ABSROOTDIR=pwd ~/src/lsif-cpp/generate-csv make O=${OUT_DIR} ${MAKE_ARGS}) set +x

if [ -n "${POST_KERNEL_BUILD_CMDS}" ]; then

7. Patched `generate-csv` in lsif-cpp, in order to make it use the FULL command, instead of just the first word of the command, and to make it pass the CC, HOSTCC to the command:

╰─[✘]$ CXX=clang++ PATH=~/src/aosp_kernel/wahoo-pie/aosp-clang/clang-r365631c/bin:$PATH ./build ++ dirname ./build

• root=.
• cd .
• yarn -s
• yarn run tsc yarn run v1.16.0

-cmd="$1" +cmd="$@"

FLAGS="-Xclang -load -Xclang $(realpath "$(dirname "${BASH_SOURCE[0]}")")/clang/libclang-index-plugin.so -Xclang -add-plugin -Xclang dxr-index -Xclang -plugin-arg-dxr-index -Xclang ." export CXX="clang++ $FLAGS" @@ -37,7 +37,7 @@ echo "Running... $cmd" CXX="$CXX" \ CC="$CC" \ DXR_CXX_CLANG_TEMP_FOLDER="$ABSOUTDIR" \

• bash -c "$cmd"

• bash -c "$cmd CC=\"$CC\" HOSTCC=clang" )

  echo

  8. Built lsif-cpp using the same clang as I'm using for the kernel, so that the clang plugin is compatible.

  pushd ~/src/lsif-cpp PATH=~/src/aosp_kernel/wahoo-pie/aosp-clang/clang-r365631c/bin:$PATH CXX=clang++ ./build popd

  9. Ran the build:

  ./build/build.sh

I've attached the output dump.lsif.zip. Note that it doesn't include the words 'read' or 'vfork', which should definitely be parts of symbols in the kernel.

I'd appreciate any direction or help you could give me...

Thanks, s1341

[chrismwendt]

Thanks for the detailed write-up 🙇 I started running through this a few days ago, but some other tasks came up. I might not get to this soon, just wanted to let you know.

[s1341]

Thanks for the update. Let me know if you need any help/info. I’m happy to run debug/development builds.