search

deepflowio / deepflow

eBPF Observability - Distributed Tracing and Profiling
https://deepflow.io
Apache License 2.0
2.99k stars 334 forks source link

关于bpf prog 问题咨询 #6196

Open Spartan-65 opened 7 months ago

Spartan-65 commented 7 months ago

我执行bpftool prog命令看到了这些bpf程序，想要请教 """61: perf_event name bpf_perf_event tag 42b497adb6d498f9 gpl"""这个程序是怎么挂载的呢？作用是什么？因为最近碰到一个perf event ringbuf 内存访问错误的问题

# bpftool prog
4: kprobe  name runtime_execute  tag 90d70e4151b22e21  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 432B  jited 282B  memlock 4096B  map_ids 13,30,27
5: kprobe  name enter_runtime_n  tag 508507df8f0510b7  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 504B  jited 336B  memlock 4096B  map_ids 13,30,29
6: kprobe  name exit_runtime_ne  tag 29e441df4186a27d  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 984B  jited 573B  memlock 4096B  map_ids 13,30,29,25
7: tracepoint  name bpf_func_sched_  tag 3b5385d5ca3c4923  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 392B  jited 256B  memlock 4096B  map_ids 13,30,18,27
8: tracepoint  name bpf_func_sched_  tag bc05acdc0a1ffcaa  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 248B  jited 177B  memlock 4096B  map_ids 13,18
9: tracepoint  name bpf_func_sys_en  tag 5568b62fd1905a97  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 976B  jited 569B  memlock 4096B  map_ids 13,6
10: tracepoint  name bpf_func_sys_ex  tag 271c699cd6661848  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 29280B  jited 18555B  memlock 32768B  map_ids 6,30,13,15,19,8,12,17,16,28
11: tracepoint  name bpf_func_sys_en  tag 6997a6a438af7c55  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 976B  jited 569B  memlock 4096B  map_ids 13,5
12: tracepoint  name bpf_func_sys_ex  tag 6a45212bc75bcd88  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 29288B  jited 18557B  memlock 32768B  map_ids 5,30,13,15,19,8,12,17,16,28
13: tracepoint  name bpf_func_sys_en  tag b059b443717b4b91  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22680B  jited 13398B  memlock 24576B  map_ids 13,7,6
14: tracepoint  name bpf_func_sys_ex  tag 93bd5c3f51a5c4ea  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 29176B  jited 18486B  memlock 32768B  map_ids 6,30,13,15,19,8,12,17,16,28
15: tracepoint  name bpf_func_sys_en  tag 81e748a081f1ff7a  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1000B  jited 586B  memlock 4096B  map_ids 13,5
16: tracepoint  name bpf_func_sys_ex  tag 2a71ee76920e9a90  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 29184B  jited 18488B  memlock 32768B  map_ids 5,30,13,15,19,8,12,17,16,28
17: kprobe  name __sys_sendmsg  tag 955c941b768968b7  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1032B  jited 628B  memlock 4096B  map_ids 13,6
18: tracepoint  name bpf_func_sys_ex  tag f5dc7dff3e9975c2  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33376B  jited 21167B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
19: kprobe  name __sys_sendmmsg  tag d5c800fea58771c0  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1080B  jited 660B  memlock 4096B  map_ids 13,6
20: tracepoint  name bpf_func_sys_ex  tag 0c0a86c366044739  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33368B  jited 21188B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
21: kprobe  name __sys_recvmsg  tag a3dbfd161a97cf25  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1056B  jited 646B  memlock 4096B  map_ids 13,5
22: tracepoint  name bpf_func_sys_ex  tag b7e2f0d26ae95d85  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33384B  jited 21166B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
23: kprobe  name __sys_recvmmsg  tag 2a63994d875b8a84  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1152B  jited 678B  memlock 4096B  map_ids 13,5
24: tracepoint  name bpf_func_sys_ex  tag eec4ed4edb0c6786  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33376B  jited 21187B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
25: kprobe  name do_writev  tag e4515a37a3b2d195  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1000B  jited 581B  memlock 4096B  map_ids 13,6
26: tracepoint  name bpf_func_sys_ex  tag 0694bd8dfa866169  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33376B  jited 21167B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
27: kprobe  name do_readv  tag 8a62658eec7ab95f  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1000B  jited 581B  memlock 4096B  map_ids 13,5
28: tracepoint  name bpf_func_sys_ex  tag 5923ee41b6f8c668  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33384B  jited 21166B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
29: tracepoint  name bpf_func_sys_en  tag 3951d66b5c406e63  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 14112B  jited 7966B  memlock 16384B  map_ids 13,7,19,5,24,20
30: tracepoint  name bpf_func_sys_ex  tag 33609d4bdd97b5a4  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 2344B  jited 1462B  memlock 4096B  map_ids 5,13,22,9,15
31: tracepoint  name bpf_func_sys_en  tag 9b9052a01064d564  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 456B  jited 314B  memlock 4096B  map_ids 9,18
32: tracepoint  name bpf_func_sys_ex  tag a5a8e89f4e9161b6  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 992B  jited 620B  memlock 4096B  map_ids 23,19,24
33: tracepoint  name bpf_func_sys_ex  tag 099c054eb523d1c0  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 168B  jited 137B  memlock 4096B  map_ids 20
34: tracepoint  name bpf_func_sys_ex  tag 099c054eb523d1c0  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 168B  jited 137B  memlock 4096B  map_ids 20
35: tracepoint  name bpf_func_sys_en  tag 0c77e12b0c5b5e23  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 168B  jited 137B  memlock 4096B  map_ids 20
36: tracepoint  name bpf_prog_tp__ou  tag 6ea3a0a11a294595  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 11120B  jited 6125B  memlock 12288B  map_ids 9,5,6,11,18
37: kprobe  name bpf_prog_kp__ou  tag 6ea3a0a11a294595  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 11120B  jited 6125B  memlock 12288B  map_ids 9,5,6,11,18
38: tracepoint  name bpf_prog_tp__pr  tag 6819113e0d1444d3  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 26392B  jited 17177B  memlock 28672B  map_ids 8,19,17,5,6,16,15
39: kprobe  name bpf_prog_kp__pr  tag 6819113e0d1444d3  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 26392B  jited 17177B  memlock 28672B  map_ids 8,19,17,5,6,16,14
40: tracepoint  name bpf_prog_tp__da  tag 785f606d1f9530bf  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 15040B  jited 8905B  memlock 16384B  map_ids 8,19,6,5,24,15,22,27,26,25,23,9,13,20
41: kprobe  name bpf_prog_kp__da  tag d79783c6940899a5  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 15112B  jited 8995B  memlock 16384B  map_ids 8,19,6,5,24,22,27,26,25,23,9,13,14,20
42: tracepoint  name bpf_prog_tp__io  tag 3c37195612772952  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 13944B  jited 7801B  memlock 16384B  map_ids 5,22,27,6,26,25,23,13,11,9,15
43: kprobe  name uprobe_go_tls_w  tag 6f6fa8145ac5bbaa  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 5016B  jited 2787B  memlock 8192B  map_ids 30,10,13,27,31
44: kprobe  name uprobe_go_tls_w  tag d637b901d45011ec  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 27568B  jited 17652B  memlock 28672B  map_ids 30,27,31,6,13,19,8,12,17,16,14,28
45: kprobe  name uprobe_go_tls_r  tag 8580da5597b46608  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 5016B  jited 2795B  memlock 8192B  map_ids 30,10,13,27,31
46: kprobe  name uprobe_go_tls_r  tag 41daa736adf10af5  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 28536B  jited 18255B  memlock 28672B  map_ids 30,27,31,13,28,5,19,8,12,17,16,14
47: kprobe  name uprobe_go_http2  tag 5a7b5c44527e36e6  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 14280B  jited 8025B  memlock 16384B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
48: kprobe  name uprobe_go_http2  tag 56fda62222a21045  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 13848B  jited 7695B  memlock 16384B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
49: kprobe  name uprobe_go_http2  tag ef874244e219635a  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22008B  jited 13201B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
50: kprobe  name uprobe_go_http2  tag 11ece519ef186152  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 18224B  jited 10718B  memlock 20480B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
51: kprobe  name uprobe_go_http2  tag 825fb7b70e966a7f  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22040B  jited 13222B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
52: kprobe  name uprobe_go_loopy  tag 2ec1f68f1c2df330  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 24088B  jited 14134B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
53: kprobe  name uprobe_go_http2  tag bc3926591c97b90c  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22232B  jited 13350B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
54: kprobe  name uprobe_go_http2  tag 950f179507243a4f  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22232B  jited 13350B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
55: kprobe  name uprobe_golang_o  tag 130d4399f8d639b5  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 6488B  jited 3618B  memlock 8192B  map_ids 13,30,10,22,24,19,18
56: kprobe  name uprobe_golang_o  tag f5c26e66db3f2fac  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 6312B  jited 3516B  memlock 8192B  map_ids 13,30,10,22,24,19,18
57: kprobe  name uprobe_openssl_  tag 89c1f779d8e5568e  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1160B  jited 653B  memlock 4096B  map_ids 13,21
58: kprobe  name uprobe_openssl_  tag 0f5542801a1e60ac  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 27648B  jited 17408B  memlock 28672B  map_ids 21,6,13,19,8,12,17,16,14,30,28
59: kprobe  name uprobe_openssl_  tag 0ba94b3749b7d3f2  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1160B  jited 653B  memlock 4096B  map_ids 13,21
60: kprobe  name uprobe_openssl_  tag b2436988f47b4557  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 27656B  jited 17410B  memlock 28672B  map_ids 21,5,13,19,8,12,17,16,14,30,28
61: perf_event  name bpf_perf_event  tag 42b497adb6d498f9  gpl
    loaded_at 2024-04-22T17:06:22+0800  uid 0
    xlated 2072B  jited 1133B  memlock 4096B  map_ids 34,35,32,36,33
yinjiping commented 7 months ago

@Spartan-65 你好

perf_event ： 周期性采样的事件触发时会执行这个programe (perf_event), 主要用于获取函数栈信息然后推送给用户层处理。

相关load, attach, buffer reader可看下面关联代码。

yinjiping commented 7 months ago

@Spartan-65 您好，这个问题大概是这个内存泄露造成的，可查看这个修复 https://github.com/deepflowio/deepflow/pull/6218

Spartan-65 commented 7 months ago

@yinjiping 谢谢，我编译测试一下看有没有问题了

Spartan-65 commented 7 months ago

我执行bpftool prog命令看到了这些bpf程序，想要请教 """61: perf_event name bpf_perf_event tag 42b497adb6d498f9 gpl"""这个程序是怎么挂载的呢？作用是什么？因为最近碰到一个perf event ringbuf 内存访问错误的问题

# bpftool prog
4: kprobe  name runtime_execute  tag 90d70e4151b22e21  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 432B  jited 282B  memlock 4096B  map_ids 13,30,27
5: kprobe  name enter_runtime_n  tag 508507df8f0510b7  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 504B  jited 336B  memlock 4096B  map_ids 13,30,29
6: kprobe  name exit_runtime_ne  tag 29e441df4186a27d  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 984B  jited 573B  memlock 4096B  map_ids 13,30,29,25
7: tracepoint  name bpf_func_sched_  tag 3b5385d5ca3c4923  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 392B  jited 256B  memlock 4096B  map_ids 13,30,18,27
8: tracepoint  name bpf_func_sched_  tag bc05acdc0a1ffcaa  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 248B  jited 177B  memlock 4096B  map_ids 13,18
9: tracepoint  name bpf_func_sys_en  tag 5568b62fd1905a97  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 976B  jited 569B  memlock 4096B  map_ids 13,6
10: tracepoint  name bpf_func_sys_ex  tag 271c699cd6661848  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 29280B  jited 18555B  memlock 32768B  map_ids 6,30,13,15,19,8,12,17,16,28
11: tracepoint  name bpf_func_sys_en  tag 6997a6a438af7c55  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 976B  jited 569B  memlock 4096B  map_ids 13,5
12: tracepoint  name bpf_func_sys_ex  tag 6a45212bc75bcd88  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 29288B  jited 18557B  memlock 32768B  map_ids 5,30,13,15,19,8,12,17,16,28
13: tracepoint  name bpf_func_sys_en  tag b059b443717b4b91  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 22680B  jited 13398B  memlock 24576B  map_ids 13,7,6
14: tracepoint  name bpf_func_sys_ex  tag 93bd5c3f51a5c4ea  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 29176B  jited 18486B  memlock 32768B  map_ids 6,30,13,15,19,8,12,17,16,28
15: tracepoint  name bpf_func_sys_en  tag 81e748a081f1ff7a  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1000B  jited 586B  memlock 4096B  map_ids 13,5
16: tracepoint  name bpf_func_sys_ex  tag 2a71ee76920e9a90  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 29184B  jited 18488B  memlock 32768B  map_ids 5,30,13,15,19,8,12,17,16,28
17: kprobe  name __sys_sendmsg  tag 955c941b768968b7  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1032B  jited 628B  memlock 4096B  map_ids 13,6
18: tracepoint  name bpf_func_sys_ex  tag f5dc7dff3e9975c2  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 33376B  jited 21167B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
19: kprobe  name __sys_sendmmsg  tag d5c800fea58771c0  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1080B  jited 660B  memlock 4096B  map_ids 13,6
20: tracepoint  name bpf_func_sys_ex  tag 0c0a86c366044739  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 33368B  jited 21188B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
21: kprobe  name __sys_recvmsg  tag a3dbfd161a97cf25  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1056B  jited 646B  memlock 4096B  map_ids 13,5
22: tracepoint  name bpf_func_sys_ex  tag b7e2f0d26ae95d85  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 33384B  jited 21166B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
23: kprobe  name __sys_recvmmsg  tag 2a63994d875b8a84  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1152B  jited 678B  memlock 4096B  map_ids 13,5
24: tracepoint  name bpf_func_sys_ex  tag eec4ed4edb0c6786  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 33376B  jited 21187B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
25: kprobe  name do_writev  tag e4515a37a3b2d195  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1000B  jited 581B  memlock 4096B  map_ids 13,6
26: tracepoint  name bpf_func_sys_ex  tag 0694bd8dfa866169  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 33376B  jited 21167B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
27: kprobe  name do_readv  tag 8a62658eec7ab95f  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1000B  jited 581B  memlock 4096B  map_ids 13,5
28: tracepoint  name bpf_func_sys_ex  tag 5923ee41b6f8c668  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 33384B  jited 21166B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
29: tracepoint  name bpf_func_sys_en  tag 3951d66b5c406e63  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 14112B  jited 7966B  memlock 16384B  map_ids 13,7,19,5,24,20
30: tracepoint  name bpf_func_sys_ex  tag 33609d4bdd97b5a4  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 2344B  jited 1462B  memlock 4096B  map_ids 5,13,22,9,15
31: tracepoint  name bpf_func_sys_en  tag 9b9052a01064d564  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 456B  jited 314B  memlock 4096B  map_ids 9,18
32: tracepoint  name bpf_func_sys_ex  tag a5a8e89f4e9161b6  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 992B  jited 620B  memlock 4096B  map_ids 23,19,24
33: tracepoint  name bpf_func_sys_ex  tag 099c054eb523d1c0  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 168B  jited 137B  memlock 4096B  map_ids 20
34: tracepoint  name bpf_func_sys_ex  tag 099c054eb523d1c0  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 168B  jited 137B  memlock 4096B  map_ids 20
35: tracepoint  name bpf_func_sys_en  tag 0c77e12b0c5b5e23  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 168B  jited 137B  memlock 4096B  map_ids 20
36: tracepoint  name bpf_prog_tp__ou  tag 6ea3a0a11a294595  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 11120B  jited 6125B  memlock 12288B  map_ids 9,5,6,11,18
37: kprobe  name bpf_prog_kp__ou  tag 6ea3a0a11a294595  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 11120B  jited 6125B  memlock 12288B  map_ids 9,5,6,11,18
38: tracepoint  name bpf_prog_tp__pr  tag 6819113e0d1444d3  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 26392B  jited 17177B  memlock 28672B  map_ids 8,19,17,5,6,16,15
39: kprobe  name bpf_prog_kp__pr  tag 6819113e0d1444d3  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 26392B  jited 17177B  memlock 28672B  map_ids 8,19,17,5,6,16,14
40: tracepoint  name bpf_prog_tp__da  tag 785f606d1f9530bf  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 15040B  jited 8905B  memlock 16384B  map_ids 8,19,6,5,24,15,22,27,26,25,23,9,13,20
41: kprobe  name bpf_prog_kp__da  tag d79783c6940899a5  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 15112B  jited 8995B  memlock 16384B  map_ids 8,19,6,5,24,22,27,26,25,23,9,13,14,20
42: tracepoint  name bpf_prog_tp__io  tag 3c37195612772952  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 13944B  jited 7801B  memlock 16384B  map_ids 5,22,27,6,26,25,23,13,11,9,15
43: kprobe  name uprobe_go_tls_w  tag 6f6fa8145ac5bbaa  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 5016B  jited 2787B  memlock 8192B  map_ids 30,10,13,27,31
44: kprobe  name uprobe_go_tls_w  tag d637b901d45011ec  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 27568B  jited 17652B  memlock 28672B  map_ids 30,27,31,6,13,19,8,12,17,16,14,28
45: kprobe  name uprobe_go_tls_r  tag 8580da5597b46608  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 5016B  jited 2795B  memlock 8192B  map_ids 30,10,13,27,31
46: kprobe  name uprobe_go_tls_r  tag 41daa736adf10af5  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 28536B  jited 18255B  memlock 28672B  map_ids 30,27,31,13,28,5,19,8,12,17,16,14
47: kprobe  name uprobe_go_http2  tag 5a7b5c44527e36e6  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 14280B  jited 8025B  memlock 16384B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
48: kprobe  name uprobe_go_http2  tag 56fda62222a21045  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 13848B  jited 7695B  memlock 16384B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
49: kprobe  name uprobe_go_http2  tag ef874244e219635a  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 22008B  jited 13201B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
50: kprobe  name uprobe_go_http2  tag 11ece519ef186152  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 18224B  jited 10718B  memlock 20480B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
51: kprobe  name uprobe_go_http2  tag 825fb7b70e966a7f  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 22040B  jited 13222B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
52: kprobe  name uprobe_go_loopy  tag 2ec1f68f1c2df330  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 24088B  jited 14134B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
53: kprobe  name uprobe_go_http2  tag bc3926591c97b90c  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 22232B  jited 13350B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
54: kprobe  name uprobe_go_http2  tag 950f179507243a4f  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 22232B  jited 13350B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
55: kprobe  name uprobe_golang_o  tag 130d4399f8d639b5  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 6488B  jited 3618B  memlock 8192B  map_ids 13,30,10,22,24,19,18
56: kprobe  name uprobe_golang_o  tag f5c26e66db3f2fac  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 6312B  jited 3516B  memlock 8192B  map_ids 13,30,10,22,24,19,18
57: kprobe  name uprobe_openssl_  tag 89c1f779d8e5568e  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1160B  jited 653B  memlock 4096B  map_ids 13,21
58: kprobe  name uprobe_openssl_  tag 0f5542801a1e60ac  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 27648B  jited 17408B  memlock 28672B  map_ids 21,6,13,19,8,12,17,16,14,30,28
59: kprobe  name uprobe_openssl_  tag 0ba94b3749b7d3f2  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 1160B  jited 653B  memlock 4096B  map_ids 13,21
60: kprobe  name uprobe_openssl_  tag b2436988f47b4557  gpl
  loaded_at 2024-04-22T17:06:21+0800  uid 0
  xlated 27656B  jited 17410B  memlock 28672B  map_ids 21,5,13,19,8,12,17,16,14,30,28
61: perf_event  name bpf_perf_event  tag 42b497adb6d498f9  gpl
  loaded_at 2024-04-22T17:06:22+0800  uid 0
  xlated 2072B  jited 1133B  memlock 4096B  map_ids 34,35,32,36,33

抱歉，我之前没有没表达清楚。 情况是这样: 我在部署了deepflow的集群node上运行falco(也是使用perf event机制收集内核事件)，会使得falco 通过perf_event_open打开的fd再映射通过mmap映射到内存上的ringbuf 异常

falco会在这里出现退出，这块是读取的data_len 太大了
scap_next buffer corruption

所以我想问的是这个perf event 的bpf函数是否会对其他程序使用perf event机制产生影响?

yinjiping commented 7 months ago

我执行bpftool prog命令看到了这些bpf程序，想要请教 """61: perf_event name bpf_perf_event tag 42b497adb6d498f9 gpl"""这个程序是怎么挂载的呢？作用是什么？因为最近碰到一个perf event ringbuf 内存访问错误的问题

# bpftool prog
4: kprobe  name runtime_execute  tag 90d70e4151b22e21  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 432B  jited 282B  memlock 4096B  map_ids 13,30,27
5: kprobe  name enter_runtime_n  tag 508507df8f0510b7  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 504B  jited 336B  memlock 4096B  map_ids 13,30,29
6: kprobe  name exit_runtime_ne  tag 29e441df4186a27d  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 984B  jited 573B  memlock 4096B  map_ids 13,30,29,25
7: tracepoint  name bpf_func_sched_  tag 3b5385d5ca3c4923  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 392B  jited 256B  memlock 4096B  map_ids 13,30,18,27
8: tracepoint  name bpf_func_sched_  tag bc05acdc0a1ffcaa  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 248B  jited 177B  memlock 4096B  map_ids 13,18
9: tracepoint  name bpf_func_sys_en  tag 5568b62fd1905a97  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 976B  jited 569B  memlock 4096B  map_ids 13,6
10: tracepoint  name bpf_func_sys_ex  tag 271c699cd6661848  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 29280B  jited 18555B  memlock 32768B  map_ids 6,30,13,15,19,8,12,17,16,28
11: tracepoint  name bpf_func_sys_en  tag 6997a6a438af7c55  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 976B  jited 569B  memlock 4096B  map_ids 13,5
12: tracepoint  name bpf_func_sys_ex  tag 6a45212bc75bcd88  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 29288B  jited 18557B  memlock 32768B  map_ids 5,30,13,15,19,8,12,17,16,28
13: tracepoint  name bpf_func_sys_en  tag b059b443717b4b91  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22680B  jited 13398B  memlock 24576B  map_ids 13,7,6
14: tracepoint  name bpf_func_sys_ex  tag 93bd5c3f51a5c4ea  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 29176B  jited 18486B  memlock 32768B  map_ids 6,30,13,15,19,8,12,17,16,28
15: tracepoint  name bpf_func_sys_en  tag 81e748a081f1ff7a  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1000B  jited 586B  memlock 4096B  map_ids 13,5
16: tracepoint  name bpf_func_sys_ex  tag 2a71ee76920e9a90  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 29184B  jited 18488B  memlock 32768B  map_ids 5,30,13,15,19,8,12,17,16,28
17: kprobe  name __sys_sendmsg  tag 955c941b768968b7  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1032B  jited 628B  memlock 4096B  map_ids 13,6
18: tracepoint  name bpf_func_sys_ex  tag f5dc7dff3e9975c2  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33376B  jited 21167B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
19: kprobe  name __sys_sendmmsg  tag d5c800fea58771c0  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1080B  jited 660B  memlock 4096B  map_ids 13,6
20: tracepoint  name bpf_func_sys_ex  tag 0c0a86c366044739  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33368B  jited 21188B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
21: kprobe  name __sys_recvmsg  tag a3dbfd161a97cf25  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1056B  jited 646B  memlock 4096B  map_ids 13,5
22: tracepoint  name bpf_func_sys_ex  tag b7e2f0d26ae95d85  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33384B  jited 21166B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
23: kprobe  name __sys_recvmmsg  tag 2a63994d875b8a84  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1152B  jited 678B  memlock 4096B  map_ids 13,5
24: tracepoint  name bpf_func_sys_ex  tag eec4ed4edb0c6786  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33376B  jited 21187B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
25: kprobe  name do_writev  tag e4515a37a3b2d195  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1000B  jited 581B  memlock 4096B  map_ids 13,6
26: tracepoint  name bpf_func_sys_ex  tag 0694bd8dfa866169  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33376B  jited 21167B  memlock 36864B  map_ids 6,30,13,15,19,8,12,17,16,28
27: kprobe  name do_readv  tag 8a62658eec7ab95f  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1000B  jited 581B  memlock 4096B  map_ids 13,5
28: tracepoint  name bpf_func_sys_ex  tag 5923ee41b6f8c668  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 33384B  jited 21166B  memlock 36864B  map_ids 5,30,13,15,19,8,12,17,16,28
29: tracepoint  name bpf_func_sys_en  tag 3951d66b5c406e63  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 14112B  jited 7966B  memlock 16384B  map_ids 13,7,19,5,24,20
30: tracepoint  name bpf_func_sys_ex  tag 33609d4bdd97b5a4  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 2344B  jited 1462B  memlock 4096B  map_ids 5,13,22,9,15
31: tracepoint  name bpf_func_sys_en  tag 9b9052a01064d564  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 456B  jited 314B  memlock 4096B  map_ids 9,18
32: tracepoint  name bpf_func_sys_ex  tag a5a8e89f4e9161b6  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 992B  jited 620B  memlock 4096B  map_ids 23,19,24
33: tracepoint  name bpf_func_sys_ex  tag 099c054eb523d1c0  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 168B  jited 137B  memlock 4096B  map_ids 20
34: tracepoint  name bpf_func_sys_ex  tag 099c054eb523d1c0  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 168B  jited 137B  memlock 4096B  map_ids 20
35: tracepoint  name bpf_func_sys_en  tag 0c77e12b0c5b5e23  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 168B  jited 137B  memlock 4096B  map_ids 20
36: tracepoint  name bpf_prog_tp__ou  tag 6ea3a0a11a294595  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 11120B  jited 6125B  memlock 12288B  map_ids 9,5,6,11,18
37: kprobe  name bpf_prog_kp__ou  tag 6ea3a0a11a294595  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 11120B  jited 6125B  memlock 12288B  map_ids 9,5,6,11,18
38: tracepoint  name bpf_prog_tp__pr  tag 6819113e0d1444d3  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 26392B  jited 17177B  memlock 28672B  map_ids 8,19,17,5,6,16,15
39: kprobe  name bpf_prog_kp__pr  tag 6819113e0d1444d3  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 26392B  jited 17177B  memlock 28672B  map_ids 8,19,17,5,6,16,14
40: tracepoint  name bpf_prog_tp__da  tag 785f606d1f9530bf  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 15040B  jited 8905B  memlock 16384B  map_ids 8,19,6,5,24,15,22,27,26,25,23,9,13,20
41: kprobe  name bpf_prog_kp__da  tag d79783c6940899a5  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 15112B  jited 8995B  memlock 16384B  map_ids 8,19,6,5,24,22,27,26,25,23,9,13,14,20
42: tracepoint  name bpf_prog_tp__io  tag 3c37195612772952  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 13944B  jited 7801B  memlock 16384B  map_ids 5,22,27,6,26,25,23,13,11,9,15
43: kprobe  name uprobe_go_tls_w  tag 6f6fa8145ac5bbaa  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 5016B  jited 2787B  memlock 8192B  map_ids 30,10,13,27,31
44: kprobe  name uprobe_go_tls_w  tag d637b901d45011ec  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 27568B  jited 17652B  memlock 28672B  map_ids 30,27,31,6,13,19,8,12,17,16,14,28
45: kprobe  name uprobe_go_tls_r  tag 8580da5597b46608  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 5016B  jited 2795B  memlock 8192B  map_ids 30,10,13,27,31
46: kprobe  name uprobe_go_tls_r  tag 41daa736adf10af5  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 28536B  jited 18255B  memlock 28672B  map_ids 30,27,31,13,28,5,19,8,12,17,16,14
47: kprobe  name uprobe_go_http2  tag 5a7b5c44527e36e6  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 14280B  jited 8025B  memlock 16384B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
48: kprobe  name uprobe_go_http2  tag 56fda62222a21045  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 13848B  jited 7695B  memlock 16384B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
49: kprobe  name uprobe_go_http2  tag ef874244e219635a  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22008B  jited 13201B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
50: kprobe  name uprobe_go_http2  tag 11ece519ef186152  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 18224B  jited 10718B  memlock 20480B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
51: kprobe  name uprobe_go_http2  tag 825fb7b70e966a7f  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22040B  jited 13222B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
52: kprobe  name uprobe_go_loopy  tag 2ec1f68f1c2df330  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 24088B  jited 14134B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
53: kprobe  name uprobe_go_http2  tag bc3926591c97b90c  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22232B  jited 13350B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
54: kprobe  name uprobe_go_http2  tag 950f179507243a4f  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 22232B  jited 13350B  memlock 24576B  map_ids 13,30,10,28,22,24,19,27,26,25,23,18
55: kprobe  name uprobe_golang_o  tag 130d4399f8d639b5  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 6488B  jited 3618B  memlock 8192B  map_ids 13,30,10,22,24,19,18
56: kprobe  name uprobe_golang_o  tag f5c26e66db3f2fac  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 6312B  jited 3516B  memlock 8192B  map_ids 13,30,10,22,24,19,18
57: kprobe  name uprobe_openssl_  tag 89c1f779d8e5568e  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1160B  jited 653B  memlock 4096B  map_ids 13,21
58: kprobe  name uprobe_openssl_  tag 0f5542801a1e60ac  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 27648B  jited 17408B  memlock 28672B  map_ids 21,6,13,19,8,12,17,16,14,30,28
59: kprobe  name uprobe_openssl_  tag 0ba94b3749b7d3f2  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 1160B  jited 653B  memlock 4096B  map_ids 13,21
60: kprobe  name uprobe_openssl_  tag b2436988f47b4557  gpl
    loaded_at 2024-04-22T17:06:21+0800  uid 0
    xlated 27656B  jited 17410B  memlock 28672B  map_ids 21,5,13,19,8,12,17,16,14,30,28
61: perf_event  name bpf_perf_event  tag 42b497adb6d498f9  gpl
    loaded_at 2024-04-22T17:06:22+0800  uid 0
    xlated 2072B  jited 1133B  memlock 4096B  map_ids 34,35,32,36,33

抱歉，我之前没有没表达清楚。 情况是这样: 我在部署了deepflow的集群node上运行falco(也是使用perf event机制收集内核事件)，会使得falco 通过perf_event_open打开的fd再映射通过mmap映射到内存上的ringbuf 异常

falco会在这里出现退出，这块是读取的data_len 太大了 scap_next buffer corruption

所以我想问的是这个perf event 的bpf函数是否会对其他程序使用perf event机制产生影响?

@Spartan-65 deepflow agent实际调用的attach接口在这里 https://github.com/iovisor/bcc/blob/679166bdee74302b46b14c3a8fe5c3db7198d3f4/src/cc/libbpf.c#L1729 实际上我们知道：syscall(__NR_perf_event_open） 会生成一个事件的fd, 这个事件会被kernel perf子系统捕获，并交给事先绑定的eBPF prog 处理。(事件和eBPF prog 会通过ioctl(event_fd, PERF_EVENT_IOC_SET_BPF, progfd)绑定)。 可以有多个采样事件并互不影响（不同的事件由各自指定eBPF prog处理）。 perf ringbuffer (BPF_MAP_TYPE_PERF_EVENT_ARRAY)， 这种buffer是deepflow自身创建的内存和perf buffer关联 这一部分的底层接口 https://github.com/iovisor/bcc/blob/679166bdee74302b46b14c3a8fe5c3db7198d3f4/src/cc/libbpf.c#L1601

mmap(NULL, mmap_size, PROT_READ | PROT_WRITE, MAP_SHARED, reader->fd, 0)  // reader->fd 是perf buffer的perCPU 事件fd

不会对其他程序产生影响，这是Linux kernel perf子系统机制保障的。

另外，您这边提到的ringbuffer 应该是 BPF_MAP_TYPE_RINGBUF 这种类型的map， 它和我们使用的eBPF perfbuf是完全不一样的东西，deepflow agent没有使用BPF_MAP_TYPE_RINGBUF 这种类型的map。

Spartan-65 commented 7 months ago

@yinjiping falco 使用的也是 BPF_MAP_TYPE_PERF_EVENT_ARRAY，使用方式基本上是一样的，但是falco不会有这个bpf prog，perf event机制应该是内核实现的，不用加载perf_event的bpf prog。意思是perf event 的bpf 程序是bcc加载的？
perf_event_fd: https://github.com/falcosecurity/libs/blob/master/userspace/libscap/engine/bpf/scap_bpf.c#L1607 mmap映射: https://github.com/falcosecurity/libs/blob/master/userspace/libscap/engine/bpf/scap_bpf.c#L1629 环形队列 error的位置: https://github.com/falcosecurity/libs/blob/master/userspace/libscap/ringbuffer/ringbuffer.h#L272

在error的位置部分打印信息

pe->len: 2896438784, pe->type: 35682, pe->nparams:12528, dev->m_sn_len: 20696 dev->m_lastreadsize: 57118428
Syscall event drop monitoring:
   - event drop detected: 0 occurrences
   - num times actions taken: 0
2024-04-26T09:38:57+0000: Closing event source 'syscall'
Events detected: 0
Rule counts by severity:
Triggered rules by rule name:
Error: scap_next buffer corruption

这个pe->type范围在几百以内，说明这个队列指针偏移可能错了，不知道为什么在装了deepflow的节点上这块会出错，挺奇怪的

sharang commented 7 months ago

方便找到它，我先打开。

yinjiping commented 7 months ago

@yinjiping falco 使用的也是 BPF_MAP_TYPE_PERF_EVENT_ARRAY，使用方式基本上是一样的，但是falco不会有这个bpf prog，perf event机制应该是内核实现的，不用加载perf_event的bpf prog。意思是perf event 的bpf 程序是bcc加载的？ perf_event_fd: https://github.com/falcosecurity/libs/blob/master/userspace/libscap/engine/bpf/scap_bpf.c#L1607 mmap映射: https://github.com/falcosecurity/libs/blob/master/userspace/libscap/engine/bpf/scap_bpf.c#L1629 环形队列 error的位置: https://github.com/falcosecurity/libs/blob/master/userspace/libscap/ringbuffer/ringbuffer.h#L272

在error的位置部分打印信息

pe->len: 2896438784, pe->type: 35682, pe->nparams:12528, dev->m_sn_len: 20696 dev->m_lastreadsize: 57118428
Syscall event drop monitoring:
   - event drop detected: 0 occurrences
   - num times actions taken: 0
2024-04-26T09:38:57+0000: Closing event source 'syscall'
Events detected: 0
Rule counts by severity:
Triggered rules by rule name:
Error: scap_next buffer corruption

这个pe->type范围在几百以内，说明这个队列指针偏移可能错了，不知道为什么在装了deepflow的节点上这块会出错，挺奇怪的

@Spartan-65 想和您确认一下，这个和deepflow agent正相关吗？ 即： 在这个节点上，启用deepflow agent会有存在问题，而把deepflow agent停用后不会出现您描述的问题，是这样吗？ 另外，Linux系统是什么，具体的内核版本是多少？

Spartan-65 commented 7 months ago

@yinjiping 目前看是的，有deepflow的节点falco就会出错

uname -a
Linux bclinux-test 4.19.25-204.el7.bclinux.x86_64 #1 SMP Wed Dec 23 15:41:17 CST 2020 x86_64 x86_64 x86_64 GNU/Linux

可以单点部署falco ebpf模式测试

docker run --pull=always -i -t --rm --privileged -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro -v /etc:/host/etc -e FALCO_DRIVER_LOADER_OPTIONS=ebpf  falcosecurity/falco