Proposal: Improve accuracy of bpf prog info

Asphaltt commented 1 month ago

When I deep dive into struct bpf_prog_info, I find that we can retrieve ksyms and ksyms' corresponding func info by ProgramInfo (see ebpf PR info: expose ksym info and func info by ProgramInfo).

Q: For `--output-stack`, what is the exact bpf prog tracked?

For example, use one tc-bpf code for multiple pods in k8s env:

# ./pwru --output-meta --output-tuple --output-stack --filter-trace-tc --filter-func '.*udp.*' --output-limit-lines 10 icmp
2024/10/09 10:09:06 Attaching tc-bpf progs...
2024/10/09 10:09:07 Attaching kprobes (via kprobe)...
38 / 38 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 478 p/s
2024/10/09 10:09:07 Attached (ignored 1)
2024/10/09 10:09:07 Listening for events..
SKB                CPU PROCESS          NETNS      MARK/x        IFACE       PROTO  MTU   LEN   TUPLE FUNC
0xffff92671be13d00 20  coredns:21719    4026531840 0        ~3bad1d0518aa:40 0x0800 1430  128   10.x.y.z:0->192.168.0.10:0(icmp)     bpf_prog_f641265f228ac785_XXX[bpf](tc)
sctp_init[sctp]
sctp_init[sctp]
sctp_init[sctp]
sctp_init[sctp]
bpf_prog_f641265f228ac785_XXX[bpf]
tcf_classify
__netif_receive_skb_core
__netif_receive_skb_one_core
__netif_receive_skb

# grep -c bpf_prog_f641265f228ac785 /proc/kallsyms
29

And, what if pwru --output-stack --filter-track-bpf-helpers?

# ./pwru --output-meta --output-tuple --output-stack --filter-trace-tc --filter-track-bpf-helpers --filter-func '.*udp.*' --output-limit-lines 10 icmp
2024/10/09 10:25:12 Attaching tc-bpf progs...
2024/10/09 10:25:23 Attaching kprobes (via kprobe)...
38 / 38 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 343 p/s
2024/10/09 10:25:24 Attached (ignored 1)
2024/10/09 10:25:24 Listening for events..
SKB                CPU PROCESS          NETNS      MARK/x        IFACE       PROTO  MTU   LEN   TUPLE FUNC
0xffff9252fd663400 6   <empty>:0        4026531840 0          ens1f1np1:7    0x0800 1500  78    10.z.y.x:0->10.x.y.z:0(icmp) bpf_prog_b503cf9b4e54d648_XXX[bpf](tc)
bpf_prog_f641265f228ac785_YYY[bpf]
bpf_prog_f641265f228ac785_YYY[bpf]
knem_debug[knem]
bpf_prog_f641265f228ac785_YYY[bpf]
bpf_prog_b503cf9b4e54d648_XXX[bpf]
tcf_classify
__dev_queue_xmit
dev_queue_xmit
bond_dev_queue_xmit[bonding]
__bond_start_xmit[bonding]
bond_start_xmit[bonding]
dev_hard_start_xmit
__dev_queue_xmit
dev_queue_xmit
vlan_dev_hard_start_xmit[8021q]
dev_hard_start_xmit
__dev_queue_xmit
dev_queue_xmit

So, which bpf prog is bpf_prog_f641265f228ac785_YYY[bpf]? As there are multiple bpf_prog_f641265f228ac785_YYY[bpf] in /proc/kallsyms.

A: Provide exact bpf prog info for ksym addresses.

With the aforementioned ebpf PR, pwru is able to retrieve the exact bpf prog info for ksym addresses, even though for subprog's ksym addresses.

Then, as for subprog info for a ksym address, it can be ${subprog function name}:${prog entry function name}[${TYPE}][${ID}][bpf]. As for prog info for a ksym address, it can be ${prog entry function name}[${TYPE}][${ID}][bpf].

At the same time, the prog info for --filter-trace-tc and --filter-trace-xdp will be updated too.

TODOs

[ ] Upgrade ebpf library to latest version to retrieve ksyms info and funcs info for a bpf prog.
[ ] Prepare new prog info for bpf prog ksym addresses.

@brb @jschwinger233 WDYT?

brb commented 3 weeks ago

Thanks for the proposal. Retrieving ksym addrs from bpf_prog_info :+1:

even though for subprog's ksym addresses.

Do you mind to elaborate what do you mean by "subprog"?

Asphaltt commented 3 weeks ago

elaborate what do you mean by "subprog"?

For instance:

# less /proc/kallsyms
ffffffffc135d0d4 t bpf_prog_9cf8e6173844be22_F  [bpf]
ffffffffc135f180 t bpf_prog_ab605ebb7ccc37a8_F  [bpf]
ffffffffc13c3060 t bpf_prog_9147d89081192601_F  [bpf]
ffffffffc13c56dc t bpf_prog_96d55732a81d4096_F  [bpf]
ffffffffc135d0d4 t bpf_prog_bd58215f88be5281_kprobe_skb_3       [bpf]

ffffffffc135f180, ffffffffc13c3060 and ffffffffc13c56dc are the ksym addresses for kprobe_skb_3's subprogs. And the symbols for these subprogs are F instead of their function names.

With the aforementioned PR, pwru is able to retrieve these ksym addresses alongside the function names for these subprogs. Then, pwru is able to show the exact info for subprogs' ksym addresses.

brb commented 3 weeks ago

Got it, thanks! Yep, the proposal makes sense to me.

cilium / pwru