search

falcosecurity / libs

libsinsp, libscap, the kernel module driver, and the eBPF driver sources
https://falcosecurity.github.io/libs/
Apache License 2.0
212 stars 158 forks source link

[BUG] Compile modern_bpf error #1719

Open yzewei opened 4 months ago

yzewei commented 4 months ago

Describe the bug make drivers_test

/home/yzw/rd/libs/driver/modern_bpf/helpers/extract/extract_from_kernel.h:648:49: error: member reference base type 'struct percpu_counter[4]' is not a structure or union
  648 |                 BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~

The code files involved include : modern_bpf/programs/tail_called/events/syscall_dispatched_events/execve.bpf.c modern_bpf/programs/tail_called/events/syscall_dispatched_events/execveat.bpf.c modern_bpf/helpers/extract/extract_from_kernel.h

How to reproduce it First judge the kernel version, and then execute the corresponding operation. like this:

#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 3, 0)

instead of

if(bpf_core_field_exists(((struct kernel_cap_struct *)0)->cap)){...}

Expected behaviour

Screenshots

Environment

Additional context

### Tasks
- [ ] https://github.com/falcosecurity/libs/pull/1654
### Tasks
- [ ] add ebpf support for loongarch64
Andreagit97 commented 4 months ago

Thank you for reporting, can you provide more info?

yzewei commented 4 months ago

Thank you for reporting, can you provide more info?

  • Which kernel version are you running?
  • Which architecture are you using?
  • Can you generate the vmlinux for your system with bpftool bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h ?

Thank you for your attention! The kernel version we are currently using is:

Andreagit97 commented 4 months ago

Uhm ok looking at this compiler error 

/home/yzw/rd/libs/driver/modern_bpf/helpers/extract/extract_from_kernel.h:648:49: error: member reference base type 'struct percpu_counter[4]' is not a structure or union
  648 |                 BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~

it seems that even if your kernel supports the new rss_stat format (so struct percpu_counter instead of struct mm_rss_stat) the struct mm_rss_stat is not removed in your kernel and for this reason, we enter in the wrong branch... we should enter in the else but according to the compiler error we are falling in the first one...

    if(bpf_core_type_exists(struct mm_rss_stat))
    {
        BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
    }
    else
    {
        struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
        BPF_CORE_READ_INTO(&swap_entries, mm_v6_2, rss_stat[MM_SWAPENTS].count);
    }

If you run these 2 commands on your machine, they should both return true, can you check it?

bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false"
yzewei commented 4 months ago

Uhm ok looking at this compiler error

/home/yzw/rd/libs/driver/modern_bpf/helpers/extract/extract_from_kernel.h:648:49: error: member reference base type 'struct percpu_counter[4]' is not a structure or union
  648 |                 BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~

it seems that even if your kernel supports the new rss_stat format (so struct percpu_counter instead of struct mm_rss_stat) the struct mm_rss_stat is not removed in your kernel and for this reason, we enter in the wrong branch... we should enter in the else but according to the compiler error we are falling in the first one...

  if(bpf_core_type_exists(struct mm_rss_stat))
  {
      BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
  }
  else
  {
      struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
      BPF_CORE_READ_INTO(&swap_entries, mm_v6_2, rss_stat[MM_SWAPENTS].count);
  }

If you run these 2 commands on your machine, they should both return true, can you check it?

bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false"

@Andreagit97 Certainly! According to the command, the result can be obtained as follows:

[yzw@fedora libs]$ bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false" 
false
true

It's true that I didn't enter the else but entered the first branch. I'm trying to clear it. This is why... In the vmlinux.h file there is really only struct percpu_counter rss_stat [4]; There was no mm_rss_stat

[yzw@fedora libs]$ find -name "vmlinux.h" | grep -rn "mm_rss_stat"
driver/modern_bpf/definitions/ppc64le/vmlinux.h:2125:struct mm_rss_stat {
driver/modern_bpf/definitions/ppc64le/vmlinux.h:2191:       struct mm_rss_stat rss_stat;
driver/modern_bpf/definitions/x86_64/vmlinux.h:1754:struct mm_rss_stat {
driver/modern_bpf/definitions/x86_64/vmlinux.h:1847:        struct mm_rss_stat rss_stat;
driver/modern_bpf/definitions/aarch64/vmlinux.h:2355:struct mm_rss_stat {
driver/modern_bpf/definitions/aarch64/vmlinux.h:2569:       struct mm_rss_stat rss_stat;
driver/modern_bpf/definitions/s390x/vmlinux.h:1668:struct mm_rss_stat {
driver/modern_bpf/definitions/s390x/vmlinux.h:1921:     struct mm_rss_stat rss_stat;
driver/modern_bpf/helpers/extract/extract_from_kernel.h:622:     * `struct mm_rss_stat` doesn't exist anymore.
driver/modern_bpf/helpers/extract/extract_from_kernel.h:624:    if(bpf_core_type_exists(struct mm_rss_stat))

[yzw@fedora loongarch64]$ find -name "vmlinux.h" | grep -rn "rss_stat"
vmlinux.h:1558:     struct percpu_counter rss_stat[4];
vmlinux.h:99637:struct trace_event_raw_rss_stat {
vmlinux.h:99668:struct trace_event_data_offsets_rss_stat {};
vmlinux.h:99690:typedef void (*btf_trace_rss_stat)(void *, struct mm_struct *, int);
yzewei commented 4 months ago

I found that the reason is that the path to percpu_counter was not imported at compile time It exists in the compiled kernel folder, but when I install it using make headers_install it is not installed at the path/usr/include/linux. And when I specify the path to percpu_counter as C_INCLUDE_PATH, there is a conflict at compiling zlib

[  1%] Performing build step for 'zlib'
In file included from ./zconf.h:450,
                 from ./zlib.h:34,
                 from test/example.c:8:
/usr/include/sys/types.h:42:18: 错误：conflicting types for ‘loff_t’; have ‘__loff_t’ {或称 ‘long int’}
   42 | typedef __loff_t loff_t;
      |                  ^~~~~~
In file included from /home/yzw/workspace/linux-chenhuacai/include/linux/limits.h:6,
                 from /usr/include/bits/local_lim.h:38,
                 from /usr/include/bits/posix1_lim.h:161,
                 from /usr/include/limits.h:195,
                 from /usr/lib/gcc/loongarch64-redhat-linux/13/include/limits.h:205,
                 from /usr/lib/gcc/loongarch64-redhat-linux/13/include/syslimits.h:7,
                 from /usr/lib/gcc/loongarch64-redhat-linux/13/include/limits.h:34,
                 from ./zconf.h:424:
/home/yzw/workspace/linux-chenhuacai/include/linux/types.h:52:33: 附注：previous declaration of ‘loff_t’ with type ‘loff_t’ {或称 ‘long long int’}
   52 | typedef __kernel_loff_t         loff_t;
      |                                 ^~~~~~
/usr/include/sys/types.h:59:17: 错误：conflicting types for ‘dev_t’; have ‘__dev_t’ {或称 ‘long unsigned int’}
   59 | typedef __dev_t dev_t;
      |                 ^~~~~
/home/yzw/workspace/linux-chenhuacai/include/linux/types.h:21:33: 附注：previous declaration of ‘dev_t’ with type ‘dev_t’ {或称 ‘unsigned int’}
   21 | typedef __kernel_dev_t          dev_t;
      |                                 ^~~~~
In file included from /usr/include/sys/types.h:130:
/usr/include/bits/types/timer_t.h:7:19: 错误：conflicting types for ‘timer_t’; have ‘__timer_t’ {或称 ‘void *’}
    7 | typedef __timer_t timer_t;
      |                   ^~~~~~~
/home/yzw/workspace/linux-chenhuacai/include/linux/types.h:31:33: 附注：previous declaration of ‘timer_t’ with type ‘timer_t’ {或称 ‘int’}
   31 | typedef __kernel_timer_t        timer_t;
      |                                 ^~~~~~~
In file included from /usr/include/sys/types.h:155:
/usr/include/bits/stdint-intn.h:27:19: 错误：conflicting types for ‘int64_t’; have ‘__int64_t’ {或称 ‘long int’}
   27 | typedef __int64_t int64_t;

What should I do about it?

Andreagit97 commented 4 months ago

Sorry, I didn't get it... if mm_rss_stat is not defined in your kernel, how is it possible that you enter the if statement?

I found that the reason is that the path to percpu_counter was not imported at compile time

I don't understand how this is related to the fact the the code finds a definition of mm_rss_stat :thinking: Could you provide the dump of your local vmlinux.h (bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h)? I don't understand who is defining this mm_rss_stat ...

yzewei commented 3 months ago

@Andreagit97 Could this be due to the Loongarch architecture being different from other architectures? Is there any way I can debug bpf_core_type_exists? Since I'm not very familiar with ebpf, I can't read the progress bar down here...

Andreagit97 commented 3 months ago

Could this be due to the Loongarch architecture being different from other architectures? Is there any way I can debug bpf_core_type_exists?

Uhm i'm not sure this is an issue with bpf_core_type_exists, in the end, this is a simple libbpf macro that calls a compiler built-in

/*
 * Convenience macro to check that provided named type
 * (struct/union/enum/typedef) exists in a target kernel.
 * Returns:
 *    1, if such type is present in target kernel's BTF;
 *    0, if no matching type is found.
 */
#define bpf_core_type_exists(type)                      \
    __builtin_preserve_type_info(*(typeof(type) *)0, BPF_TYPE_EXISTS)

The best way to debug is to obtain the vmlinux.h of your kernel and see if this type is here or not. I've seen in our previous conversation that you were able to type these commands

bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false"

Can you just run this command and provide us with the vmlinux.h?

bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h
yzewei commented 3 months ago

The root cause of the problem is the problem of the clang built-in function under the loongarch architecture

yzewei commented 3 months ago

Could this be due to the Loongarch architecture being different from other architectures? Is there any way I can debug bpf_core_type_exists?

Uhm i'm not sure this is an issue with bpf_core_type_exists, in the end, this is a simple macro that calls a compiler built-in

/*
 * Convenience macro to check that provided named type
 * (struct/union/enum/typedef) exists in a target kernel.
 * Returns:
 *    1, if such type is present in target kernel's BTF;
 *    0, if no matching type is found.
 */
#define bpf_core_type_exists(type)                        \
  __builtin_preserve_type_info(*(typeof(type) *)0, BPF_TYPE_EXISTS)

The best way to debug is to obtain the vmlinux.h of your kernel and see if this type is here or not. I've seen in our previous conversation that you were able to type these commands

bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false"

Can you just run this command and provide us with the vmlinux.h?

bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

of course，https://github.com/libbpf/libbpf-bootstrap/blob/master/vmlinux/loongarch/vmlinux_602.h

yzewei commented 3 months ago

@Andreagit97 If you would like my vmlinux.h from TCE-metal, I would be happy to provide it, please visit https://github.com/Loongson-Cloud-Community/Loongson-Cloud-Community/blob/main/docs/%E5%85%B6%E4%BB%96%E6%96%87%E6%A1%A3/vmlinux.h

os:fedora kernel-version:6.8.0-rc5 llvm-version:18 clang-version:18 BTF:on CO-RE:on

Regarding this issue, I am consulting the relevant personnel in the llvm community responsible for the loongarch architecture. If you have any suggestions, you can contact me at any time, thank you very much! Although I don't know why the built-in function has such a problem, I think you can try loongarch's vmlinux.h replication problem under x86 architecture

Andreagit97 commented 3 months ago

uhm ok i see... not sure this is the exact vmlinux of you machine but yes, in this vmlinux you provided there is no the mm_rss_stat struct, so yes this is strange... I've seen that you are using an high clang version (18), have you ever tried with a lower version? something like clang-14/clang-15? Locally i use clang-14, maybe something is changed

yzewei commented 3 months ago

uhm ok i see... not sure this is the exact vmlinux of you machine but yes, in this vmlinux you provided there is no the mm_rss_stat struct, so yes this is strange... I've seen that you are using an high clang version (18), have you ever tried with a lower version? something like clang-14/clang-15? Locally i use clang-14, maybe something is changed

Sir, I have provided the loongarch architecture vmlinux.h header file given by the libbpf official and the vmlinux.h generated by my machine respectively. Neither of these two header files from different sources mm_rss_stat structure

yzewei commented 3 months ago

use clang-14 error：

error: unable to create target: 'No available targets are compatible with triple "bpf"'
Andreagit97 commented 3 months ago

ok, I read again the issue, and we are missing something... you are compiling this on your machine so what is happening is perfectly legit, the compiler is just saying that it cannot find a definition for struct mm_rss_stat in your vmlinux.h, so we need to add it between missing definitions, sorry for that, I was convinced for some reason that you were executing the probe against your local machine...

yzewei commented 3 months ago

ok, I read again the issue, and we are missing something... you are compiling this on your machine so what is happening is perfectly legit, the compiler is just saying that it cannot find a definition for struct mm_rss_stat in your vmlinux.h, so we need to add it between missing definitions, sorry for that, I was convinced for some reason that you were executing the probe against your local machine...

Regarding this issue, if we wait for the built-in functions of clang to return to normal, it will take a long time. What are your suggestions? Add arch detection to avoid this problem? In #1654, we discussed the feasibility of switching to kernel version checking

Andreagit97 commented 3 months ago

Uhm let me explain better what is happening here. On your local machine, to compile the modern ebpf probe, you have probably added another vmlinux.h under this folder https://github.com/falcosecurity/libs/tree/0.15.0/driver/modern_bpf/definitions. In this vmlinux.h there is no definition for struct mm_rss_stat, while if you check in all the others vmlinux.h we have in this folder you will see that the struct mm_rss_stat is defined. This is because all the existing vmlinux.h are generated from a kernel version <= 6.2.

Now how can you solve this missing definition? You need to add the definition by yourself in our struct_flavors.h file. So you need to add something like

#if defined(__TARGET_ARCH_loongarch)
struct mm_rss_stat {
    atomic_long_t count[4];
};
#endif

this should solve this compilation issue, but there could be others since your vmlinux.h is generated on a recent loongarch machine...

The best solution here would be to generate a vmlinux.h for loongarch in an older machine something with a kernel version <= 6.0.0, but not sure if you have a machine to do that

yzewei commented 3 months ago

Uhm let me explain better what is happening here. On your local machine, to compile the modern ebpf probe, you have probably added another vmlinux.h under this folder https://github.com/falcosecurity/libs/tree/0.15.0/driver/modern_bpf/definitions. In this vmlinux.h there is no definition for struct mm_rss_stat, while if you check in all the others vmlinux.h we have in this folder you will see that the struct mm_rss_stat is defined. This is because all the existing vmlinux.h are generated from a kernel version <= 6.2.

Now how can you solve this missing definition? You need to add the definition by yourself in our struct_flavors.h file. So you need to add something like

#if defined(__TARGET_ARCH_loongarch)
struct mm_rss_stat {
  atomic_long_t count[4];
};
#endif

this should solve this compilation issue, but there could be others since your vmlinux.h is generated on a recent loongarch machine...

The best solution here would be to generate a vmlinux.h for loongarch in an older machine something with a kernel version <= 6.0.0, but not sure if you have a machine to do that

mm_rss_stat structure does not exist when kernel version > 6.2.0, this definition may cause problems with project compilation. I will try to compile a lower version of the kernel

yzewei commented 3 months ago

I wonder if we could change the criteria from mm_rss_stat to kernel version? like this: from

        if(bpf_core_type_exists(struct mm_rss_stat))
        {
                BPF_CORE_READ_INTO(&file_pages, mm, rss_stat.count[MM_FILEPAGES].counter);
                BPF_CORE_READ_INTO(&anon_pages, mm, rss_stat.count[MM_ANONPAGES].counter);
                BPF_CORE_READ_INTO(&shmem_pages, mm, rss_stat.count[MM_SHMEMPAGES].counter);
        }
        else
        {
                struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
                BPF_CORE_READ_INTO(&file_pages, mm_v6_2, rss_stat[MM_FILEPAGES].count);
                BPF_CORE_READ_INTO(&anon_pages, mm_v6_2, rss_stat[MM_ANONPAGES].count);
                BPF_CORE_READ_INTO(&shmem_pages, mm_v6_2, rss_stat[MM_SHMEMPAGES].count);
        }

to

#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 2, 0)
                BPF_CORE_READ_INTO(&file_pages, mm, rss_stat.count[MM_FILEPAGES].counter);
                BPF_CORE_READ_INTO(&anon_pages, mm, rss_stat.count[MM_ANONPAGES].counter);
                BPF_CORE_READ_INTO(&shmem_pages, mm, rss_stat.count[MM_SHMEMPAGES].counter);
#else
                struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
                BPF_CORE_READ_INTO(&file_pages, mm_v6_2, rss_stat[MM_FILEPAGES].count);
                BPF_CORE_READ_INTO(&anon_pages, mm_v6_2, rss_stat[MM_ANONPAGES].count);
                BPF_CORE_READ_INTO(&shmem_pages, mm_v6_2, rss_stat[MM_SHMEMPAGES].count);
#endif
Andreagit97 commented 3 months ago

I wonder if we could change the criteria from mm_rss_stat to kernel version?

Unfortunately no, because these changes could be backported in older kernel versions (so before 6.2.0), many distros do similar things... so the best way to detect these changes is to check the struct existence...

yzewei commented 3 months ago

I wonder if we could change the criteria from mm_rss_stat to kernel version?

Unfortunately no, because these changes could be backported in older kernel versions (so before 6.2.0), many distros do similar things... so the best way to detect these changes is to check the struct existence...

After debugging, it was found that there was no exception when running the bpf_core_type_exists () function. For higher kernels that do not have mm_rss_stat structures, the result returned by the loongarch platform is 0. But I still can't figure out why logic is true here

yzewei commented 3 months ago

@Andreagit97 I found that the result of judging mm_rss_stat on loongarch is correct, just need to judge under the macro definition to be normal

        int result = bpf_core_type_exists(struct mm_rss_stat);
        /* In recent kernel versions (https://github.com/torvalds/linux/commit/f1a7941243c102a44e8847e3b94ff4ff3ec56f25)
         * `struct mm_rss_stat` doesn't exist anymore.
         */
#if result == 1
                BPF_CORE_READ_INTO(&file_pages, mm, rss_stat.count[MM_FILEPAGES].counter);
                BPF_CORE_READ_INTO(&anon_pages, mm, rss_stat.count[MM_ANONPAGES].counter);
                BPF_CORE_READ_INTO(&shmem_pages, mm, rss_stat.count[MM_SHMEMPAGES].counter);
#else
                struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
                BPF_CORE_READ_INTO(&file_pages, mm_v6_2, rss_stat[MM_FILEPAGES].count);
                BPF_CORE_READ_INTO(&anon_pages, mm_v6_2, rss_stat[MM_ANONPAGES].count);
                BPF_CORE_READ_INTO(&shmem_pages, mm_v6_2, rss_stat[MM_SHMEMPAGES].count);
#endif
Andreagit97 commented 3 months ago

ei @yzewei i think you have 2 possible solutions as explained here https://github.com/falcosecurity/libs/issues/1719#issuecomment-2009125146;

  1. introduce an #if defined(__TARGET_ARCH_loongarch) to add the missing structs definitions that the compiler needs.
  2. generate an old vmlinux.h for long arch (for old I mean a kernel version <= 6.2.0, probably <= 6.0 is even better since there are other structs that are changed) and put it in the repo.

If possible I would go for solution number 2, it would be easier for everyone since it should avoid ifdefs

yzewei commented 3 months ago

ei @yzewei i think you have 2 possible solutions as explained here #1719 (comment);

  1. introduce an #if defined(__TARGET_ARCH_loongarch) to add the missing structs definitions that the compiler needs.
  2. generate an old vmlinux.h for long arch (for old I mean a kernel version <= 6.2.0, probably <= 6.0 is even better since there are other structs that are changed) and put it in the repo.

If possible I would go for solution number 2, it would be easier for everyone since it should avoid ifdefs

@Andreagit97 Compiling with vmlinux with a kernel smaller than 6.0 solved this problem, thank you for your guidance!!

yzewei commented 3 months ago
Andreagit97 commented 3 months ago

@Andreagit97 version 6.1.0 of Linux is compatible with Loongarch starting from version 5.19. So I selected this version to compile and obtain the low-version vmlinux file.

Ok now that you have this vmlinux.h file, you should take it and come back to your initial setup with kernel version 6.8.0. Now you should add this file in a new folder loongarch here https://github.com/falcosecurity/libs/tree/master/driver/modern_bpf/definitions and you should try to compile again. This time many of the initial issues should be solved, at least the one reported in this issue. If not, please report the compilation error and the vmlinux.h you generated.

Unfortunately, this version may not support the BTF of Loongarch for the time being, which resulted in some issues during compilation.

Please note that you need to only take the vmlinux.h file from the old kernel version, the compilation should happen on your actual setup (kernel version 6.8.0). If the old machine doesn't support BTF you cannot compile the modern probe on it

yzewei commented 3 months ago

@Andreagit97 version 6.1.0 of Linux is compatible with Loongarch starting from version 5.19. So I selected this version to compile and obtain the low-version vmlinux file.

Ok now that you have this vmlinux.h file, you should take it and come back to your initial setup with kernel version 6.8.0. Now you should add this file in a new folder loongarch here https://github.com/falcosecurity/libs/tree/master/driver/modern_bpf/definitions and you should try to compile again. This time many of the initial issues should be solved, at least the one reported in this issue. If not, please report the compilation error and the vmlinux.h you generated.

Unfortunately, this version may not support the BTF of Loongarch for the time being, which resulted in some issues during compilation.

Please note that you need to only take the vmlinux.h file from the old kernel version, the compilation should happen on your actual setup (kernel version 6.8.0). If the old machine doesn't support BTF you cannot compile the modern probe on it

Yes, I did it, I didn't encounter any other errors in the bpf part at present, but some data structures or data items do not exist due to kernel version problems. Under vmlinux.h, I added it in struct_flavors, let's see what problems will appear in the future

yzewei commented 3 months ago

@Andreagit97 Here's another question: 

/home/yzw/rd/libs/userspace/libpman/src/lifecycle.c: in function ‘pman_save_attached_progs’:
/home/yzw/rd/libs/userspace/libpman/src/lifecycle.c:44:76: error：‘struct <ambigous>’not member named sched_p_fork’
   44 |       g_state.attached_progs_fds[5] = bpf_program__fd(g_state.skel->progs.sched_p_fork);
      |                                                                          ^
Andreagit97 commented 3 months ago

ei @yzewei it seems like you have defined CAPTURE_SCHED_PROC_FORK somewhere for powerpc... it's difficult to say what you have done without seeing the code... can you open a PR with the changes so we can see it?

yzewei commented 3 months ago

@Andreagit97 This problem is solved because system calls can be made under loongarch: sys_exit_execve No need to add CAPTURE_SCHED_PROC_FORK

yzewei commented 3 months ago

But there is still a problem when making bpf, the following is the problem situation

In file included from /home/yzw/rd/libs/driver/bpf/probe.c:27:
/home/yzw/rd/libs/driver/bpf/fillers.h:2369:48: warning: passing 'volatile long *' to parameter of type 'long *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
 2369 |                 res = bpf_accumulate_argv_or_env(data, argv, &args_len);
      |                                                              ^~~~~~~~~
/home/yzw/rd/libs/driver/bpf/fillers.h:1923:19: note: passing argument to parameter 'args_len' here
 1923 |                                                       long *args_len)
      |                                                             ^
1 warning generated.
LLVM ERROR: Branch target out of insn range
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.  Program arguments: llc -march=bpf -filetype=obj -o /home/yzw/rd/libs/driver/bpf/probe.o /home/yzw/rd/libs/driver/bpf/probe.ll
 #0 0x00007fffe8a61acc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/lib64/libLLVM-18.so+0xa61acc)
 #1 0x00007fffe8a5f9a0 llvm::sys::RunSignalHandlers() (/lib64/libLLVM-18.so+0xa5f9a0)
 #2 0x00007fffe8a5fd88 (/lib64/libLLVM-18.so+0xa5fd88)
 #3 0x00007ffffe79c89c (linux-vdso.so.1+0x89c)
 #4 0x00007fffeffaaf58 __pthread_kill_implementation.constprop.0 (/lib64/libc.so.6+0x82f58)
 #5 0x00007fffeff651e0 raise (/lib64/libc.so.6+0x3d1e0)
 #6 0x00007fffeff502f4 abort (/lib64/libc.so.6+0x282f4)
 #7 0x00007fffe89a4d74 llvm::report_fatal_error(llvm::StringRef, bool) (/lib64/libLLVM-18.so+0x9a4d74)
 #8 0x00007fffe89a4dec llvm::install_bad_alloc_error_handler(void (*)(void*, char const*, bool), void*) (/lib64/libLLVM-18.so+0x9a4dec)
 #9 0x00007fffeb34d19c (/lib64/libLLVM-18.so+0x334d19c)
#10 0x00007fffea214eec llvm::MCAssembler::layout(llvm::MCAsmLayout&) (/lib64/libLLVM-18.so+0x2214eec)
#11 0x00007fffea215060 llvm::MCAssembler::Finish() (/lib64/libLLVM-18.so+0x2215060)
#12 0x00007fffe93ba8c4 llvm::AsmPrinter::doFinalization(llvm::Module&) (/lib64/libLLVM-18.so+0x13ba8c4)
#13 0x00007fffe8baa138 llvm::FPPassManager::doFinalization(llvm::Module&) (/lib64/libLLVM-18.so+0xbaa138)
#14 0x00007fffe8bb53e4 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/lib64/libLLVM-18.so+0xbb53e4)
#15 0x00005555587cff38 (/usr/bin/llc+0x17f38)
#16 0x00005555587c7bb4 main (/usr/bin/llc+0xfbb4)
#17 0x00007fffeff5082c __libc_start_call_main (/lib64/libc.so.6+0x2882c)
#18 0x00007fffeff50918 __libc_start_main@@GLIBC_2.36 (/lib64/libc.so.6+0x28918)
#19 0x00005555587c8140 _start (/usr/bin/llc+0x10140)
make[7]: *** [/home/yzw/rd/libs/driver/bpf/Makefile:56: /home/yzw/rd/libs/driver/bpf/probe.o]  Segmentation fault (core dumped)
make[6]: *** [/home/yzw/workspace/linux-chenhuacai/Makefile:1921：/home/yzw/rd/libs/driver/bpf] error 2
make[5]: *** [Makefile:240：__sub-make] error 2
make[4]: *** [Makefile:39：all] error 2
make[3]: *** [driver/bpf/CMakeFiles/bpf.dir/build.make:70：driver/bpf/CMakeFiles/bpf] error 2
make[2]: *** [CMakeFiles/Makefile2:859：driver/bpf/CMakeFiles/bpf.dir/all] error 2
make[1]: *** [CMakeFiles/Makefile2:866：driver/bpf/CMakeFiles/bpf.dir/rule] error 2
make: *** [Makefile:312：bpf] error 2
yzewei commented 3 months ago

ei @yzewei it seems like you have defined CAPTURE_SCHED_PROC_FORK somewhere for powerpc... it's difficult to say what you have done without seeing the code... can you open a PR with the changes so we can see it?

At present, the code needs to be sorted out and reviewed internally, so it will take some time to update the relevant code on the previous PR. Please understand, thank you very much！

yzewei commented 2 months ago

@Andreagit97 The problem has been identified, because the branch instructions compiled by llc need to encode offsets greater than 2^16 instructions, which is beyond the limit of cpuv3 and requires the use of cpuv4, and then depends on the higher version of the kernel. Other methods are being communicated so that the lower version kernel can use the bpf module of libs

Andreagit97 commented 2 months ago

oh, great catch! thank you for the update!