Documentation for enabling CONFIG_FTRACE_SYSCALLS and disabling CONFIG_TRACEFS_DISABLE_AUTOMOUNT

[droidresearch]

The documentation for getting started with android has the following instruction to make the pulsar available for android enable CONFIG_FTRACE_SYSCALLS and CONFIG_BPF_LSM disable CONFIG_TRACEFS_DISABLE_AUTOMOUNT However, starting from android 13, editing the .config file to enable/disable the config options is changed as there is a bazel based build from android-13 kernels.

Could you please point to any references on how to build the kernel with these two options enabled.

Thank you

[krsh]

For a custom kernel for Android suitable for Pulsar, we referred to this page: https://source.android.com/docs/setup/build/building-kernels#customize-config.

If you intend to build the kernel with Bazel, you can refer to this page: https://android.googlesource.com/kernel/build/+/refs/heads/master/kleaf/README.md. However, we have not yet tried this procedure.

Please let us know if you encounter any further issues.

[droidresearch]

Thanks for the response. Yes, I have been trying to figure out similar options to cusomize kernel config using bazel build. As, I am not able to make the device boot if the kernel build is customized with the traditional build config method.

Could you please let me know if you used emulator or a real device for testing the custom kernel and did you use fastboot to flash the custom kernel

[droidresearch]

Sorry accidentally closed this

[krsh]

We utilized both the emulator (x86_64 and aarch64) and real devices (aarch64 only) with Lineage, following the device-specific guides (https://wiki.lineageos.org/devices/).

[droidresearch]

Thanks, I will check with LineageOS.

[droidresearch]

I got the rest of the modules running except file monitor. the following is the error message I see in the logs. Could you help with what symbol did I miss? ERROR pulsar::pulsard::module_manager] Error in module file-system-monitor. Module stopped. ProgramLoadError { program: "lsm path_mknod", program_error: Btf(UnknownBtfTypeName { type_name: "bpf_lsm_path_mknod" }) }

[krsh]

What platform are you using (emulator and real device) and what architecture (x86_64 or aarch64)?

[droidresearch]

I have compiled a custom kernel for emulator with all the config options for emulator x86_64

vsoc_x86_64:/ # uname -a Linux localhost 5.15.104-android13-8-00025-g8dd49c100504-dirty #1 SMP PREEMPT Mon Jul 17 19:36:51 UTC 2023 x86_64 Toybox vsoc_x86_64:/ #

[krsh]

Make sure that CONFIG_BPF_LSM is enabled, and check that the output of cat /sys/kernel/security/lsm contains the string "bpf". If it does not, BPF LSM must be enabled, for example, either in Grub or directly in the kernel at compile-time.

[droidresearch]

Thanks, It is strange that I see that in /proc/config.gz (CONFIG_BPF_LSM=y) is set but do not see /sys/kernel/security/lsm

#
# BPF subsystem
#
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_JIT=y
CONFIG_BPF_JIT_ALWAYS_ON=y
CONFIG_BPF_JIT_DEFAULT_ON=y
# CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set
# CONFIG_BPF_PRELOAD is not set
CONFIG_BPF_LSM=y
# end of BPF subsystem

[droidresearch]

My bad! there securityfs was not mounted. Now, mount point is established. and I do see the lsm in /sys/kernel/security root@localhost:/data/local/tmp# ls /sys/kernel/security/ integrity lsm root@localhost:/data/local/tmp# ./pulsar-exec-x86_64-static pulsard --config-file=./pulsar.ini [2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module process-monitor [2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module file-system-monitor [2023-07-19T10:46:42Z ERROR pulsar::pulsard::module_manager] Error in module file-system-monitor. Module stopped. ProgramLoadError { program: "lsm path_mknod", program_error: Btf(UnknownBtfTypeName { type_name: "bpf_lsm_path_mknod" }) } [2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module network-monitor [2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module logger [2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module rules-engine [2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module desktop-notifier

[krsh]

You should see the content of the file /sys/kernel/security/lsm

for example, on my system it is:

$ cat /sys/kernel/security/lsm 
capability,landlock,lockdown,yama,bpf

Please check whether this file contains the string "bpf".

[droidresearch]

Ok. In my system, I see bpf but landlock, lockdown, yama is missing!! Please see the output below.

cat /sys/kernel/security/lsm capability,selinux,bpf

[krsh]

Your emulator seems to be fine with the BPF LSM configurations. Check if debugfs is mounted. If it isn't, run

mount -t debugfs none /sys/kernel/debug

[droidresearch]

Yes, the debugfs also is mounted and has the following entries

acpi    block      cleancache       devfreq           dmaengine        extfrag             hid           kfence   lru_gen_full   regmap      slab              swiotlb  virtio-ports
asoc    bluetooth  clear_warn_once  device_component  dri              f2fs                ieee80211     kprobes  opp            regulator   sleep_time        tracing  wakeup_sources
bdi     btt        clk              devices_deferred  dynamic_debug    fault_around_bytes  iio           kvm      ramdisk_pages  remoteproc  split_huge_pages  ufshcd   x86
binder  cec        cma              dma_buf           error_injection  gpio                interconnect  lru_gen  ras            sched       suspend_stats     usb
vsoc_x86_64:/data/local/tmp # 

[krsh]

ok, try also to mount tracefs, because it is checked before /sys/kernel/debug/tracing:

mount -t tracefs tracefs /sys/kernel/tracing

[droidresearch]

Ok. Please find the output of the listing the tracefs mounted to /sys/kernel/tracing

 ls -lhrt /sys/kernel/tracing/
total 0
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 set_event_pid
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 set_event_notrace_pid
-rw-rw-rw-.   1 root 3012 0 Jul 19 10:54 set_event
-r--r--r--.   1 root 3012 0 Jul 19 10:54 available_events
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 tracing_thresh
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 tracing_max_latency
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 tracing_cpumask
-r--r--r--.   1 root 3012 0 Jul 19 10:54 trace_pipe
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 trace_options
--w--w----.   1 root 3012 0 Jul 19 10:54 trace_marker_raw
--w--w--w-.   1 root 3012 0 Jul 19 10:54 trace_marker
-rw-rw-rw-.   1 root 3012 0 Jul 19 10:54 trace_clock
-rw-rw-rw-.   1 root 3012 0 Jul 19 10:54 trace
-r--r--r--.   1 root 3012 0 Jul 19 10:54 timestamp_mode
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 snapshot
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 set_graph_notrace
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 set_graph_function
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 set_ftrace_pid
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 set_ftrace_notrace_pid
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 set_ftrace_notrace
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 set_ftrace_filter
-r--r--r--.   1 root 3012 0 Jul 19 10:54 saved_tgids
-rw-rw-rw-.   1 root 3012 0 Jul 19 10:54 saved_cmdlines_size
-r--r--r--.   1 root 3012 0 Jul 19 10:54 saved_cmdlines
drwxr-xr-x.   4 root 3012 0 Jul 19 10:54 per_cpu
drwxr-xr-x.   2 root 3012 0 Jul 19 10:54 options
--w-------.   1 root 3012 0 Jul 19 10:54 free_buffer
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 error_log
-r--r--r--.   1 root 3012 0 Jul 19 10:54 enabled_functions
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 current_tracer
-r--r--r--.   1 root 3012 0 Jul 19 10:54 buffer_total_size_kb
-rw-rw-rw-.   1 root 3012 0 Jul 19 10:54 buffer_size_kb
-r--r--r--.   1 root 3012 0 Jul 19 10:54 buffer_percent
-r--r--r--.   1 root 3012 0 Jul 19 10:54 available_tracers
-r--r--r--.   1 root 3012 0 Jul 19 10:54 available_filter_functions
-r--r--r--.   1 root 3012 0 Jul 19 10:54 README
-r--r--r--.   1 root 3012 0 Jul 19 10:54 uprobe_profile
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 uprobe_events
-r--r--r--.   1 root 3012 0 Jul 19 10:54 printk_formats
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 max_graph_depth
-r--r--r--.   1 root 3012 0 Jul 19 10:54 kprobe_profile
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 kprobe_events
drwxr-xr-x.   4 root 3012 0 Jul 19 10:54 instances
-rw-r--r--.   1 root 3012 0 Jul 19 10:54 dynamic_events
-r--r--r--.   1 root 3012 0 Jul 19 10:54 dyn_ftrace_total_info
-rw-rw-rw-.   1 root 3012 0 Jul 19 10:54 tracing_on
-rw-rw-rw-.   1 root 3012 0 Jul 19 10:54 synthetic_events
drwxr-xr-x. 118 root 3012 0 Jul 19 10:54 events

[krsh]

The emulator appears to be well configured. I will try to recreate the issue as soon as possible and I will let you know.

[droidresearch]

Thank you so much for the advice so far.

[droidresearch]

I managed to fix the api_socket_path issue and got the modules running except filesystem monitor.

| MODULE              | VERSION | STATUS                                       |
+==============================================================================+
| desktop-notifier    | 0.6.0   | Running                                      |
|---------------------+---------+----------------------------------------------|
| file-system-monitor | 0.6.0   | Failed("failed program load lsm path_mknod") |
|---------------------+---------+----------------------------------------------|
| logger              | 0.6.0   | Running                                      |
|---------------------+---------+----------------------------------------------|
| network-monitor     | 0.6.0   | Running                                      |
|---------------------+---------+----------------------------------------------|
| process-monitor     | 0.6.0   | Running                                      |
|---------------------+---------+----------------------------------------------|
| rules-engine        | 0.6.0   | Running                                      |
+---------------------+---------+----------------------------------------------+

[droidresearch]

Ok. It looks like there is a regression in some version. I am testing one by one. The first version works fine (atleast the status of all modules are green but there is no monitor command so no logs)

./pulsar-exec-static pulsar status 
+---------------------+---------+---------+
| MODULE              | VERSION | STATUS  |
+=========================================+
| file-system-monitor | 0.0.1   | Running |
|---------------------+---------+---------|
| logger              | 0.0.1   | Running |
|---------------------+---------+---------|
| network-monitor     | 0.0.1   | Running |
|---------------------+---------+---------|
| process-monitor     | 0.0.1   | Running |
|---------------------+---------+---------|
| rules-engine        | 0.0.1   | Running |
|---------------------+---------+---------|
| syscall-monitor     | 0.0.1   | Running |
+---------------------+---------+---------+

[droidresearch]

I have a working environment with the following config fix to kernel

CONFIG_SECURITY_LANDLOCK=y CONFIG_SECURITY_LOCKDOWN_LSM=y CONFIG_SECURITY_YAMA=y

Thank you so much for the inputs.

The emulator appears to be well configured. I will try to recreate the issue as soon as possible and I will let you know.