Closed hezhiye closed 1 year ago
Hi
我采用容器编译后,该问题消失,但是有2个宏没有定义
ICMPV6_EXT_ECHO_REQUEST
ICMPV6_EXT_ECHO_REPLY
后来我直接把这两个删了
然后有llc -march=bpf -filetype=obj -o progs/ntrace.o In file included from progs/ntrace.c:7: /root/nettrace/shared/bpf/skb_parse.h:365:13: error: invalid application of 'typeof' to bit-field l4_proto = _C(sk, sk_protocol); ^ /root/nettrace/shared/bpf/skb_parse.h:71:20: note: expanded from macro '_C'
^
/root/nettrace/shared/bpf/skbparse.h:64:8: note: expanded from macro '' typeof(src) tmp; \ ^ /root/nettrace/shared/bpf/skb_parse.h:365:13: error: invalid application of 'sizeof' to bit-field /root/nettrace/shared/bpf/skb_parse.h:71:20: note: expanded from macro '_C'
^
/root/nettrace/shared/bpf/skbparse.h:65:36: note: expanded from macro ''
bpf_probe_read_kernel(&tmp, sizeof(src), &(src)); \
^
/root/nettrace/shared/bpf/skb_parse.h:365:13: error: address of bit-field requested
l4_proto = _C(sk, sk_protocol);
^~~~~~~
/root/nettrace/shared/bpf/skb_parse.h:71:20: note: expanded from macro '_C'
^~~~~~~~~~~
/root/nettrace/shared/bpf/skbparse.h:65:43: note: expanded from macro ''
bpf_probe_read_kernel(&tmp, sizeof(src), &(src)); \
^ ~~~
3 errors generated.
bpftool gen skeleton progs/ntrace.o > progs/ntrace.skel.h
gcc mark.c /root/nettrace/shared/pkt_utils.c /root/nettrace/component/net_utils.c /root/nettrace/component/arg_parse.c /root/nettrace/component/sys_utils.c /root/nettrace/shared/bpf_utils.c -o mark -lbpf -lelf -lz -O2 -static -I./ -I/root/nettrace/shared/bpf/ -DCOMPAT_MODE -Wno-deprecated-declarations -DVERSION=1.2.5 -DRELEASE=.tl3 -I/root/nettrace/shared/ -I/root/nettrace/component
mark.c: In function 'main':
mark.c:83:12: error: 'struct ntrace' has no member named 'data'
83 | obj->data->_bpf_args = bpf_args;
| ^~
In file included from common.h:19,
from mark.c:4:
/root/nettrace/shared/bpf_utils.h:9:53: error: 'struct ntrace' has no member named 'progs'
9 | #define BPF_PROG_FD(name) (bpf_program__fd(obj->progs.name))
| ^~
mark.c:95:23: note: in expansion of macro 'BPF_PROG_FD'
95 | if (tc_attach(BPF_PROG_FD(ntrace_mark), nic, tc_pref, false))
| ^~~
/root/nettrace/shared/bpf_utils.h:10:49: error: 'struct ntrace' has no member named 'maps'
10 | #define BPF_MAP_FD(name) (bpf_map__fd(obj->maps.name))
| ^~
mark.c:99:21: note: in expansion of macro 'BPF_MAP_FD'
99 | perf_output(BPF_MAP_FD(m_event), print_bpf_output);
| ^~~~~~
make[1]: *** [../common.mk:103: mark] Error 1
rm progs/ntrace.o
错误,但能编译出nettrace
你好,请问使用的是最新的代码吗?改成我在ubuntu20.04上尝试了一下,没有编译出错的问题。
ubuntu20.04当前使用的内核版本好像是5.4吧?这个版本是支持BTF的,不需要使用兼容模式。 请问这里是有什么特殊的原因吗?
BTW, 你的编译报错不是nettrace报错,是里面的另一个组件nodetrace报错,是不影响nettrace 的使用的。
是最新的代码,ubuntu 20.04LTS内核是5.15.0-72-generic 另外,有没有要求内核哪些配置项的文档?我用编译了aarch64的nettrace, 运行会报如下错。 root@e04-g4-single:/tmp# ./nettrace -p icmp --detail --date ERROR: failed to manually attach program prog=trace___netif_receive_skb_core, func=netif_receive_skb_core.constprop.0 ERROR: failed to manually attach program prog=trace_netif_receive_skb_core_pskb, func=netif_receive_skb_core.constprop.0 ERROR: failed to manually attach program prog=trace_ip_rcv_core, func=ip_rcv_core.constprop.0 ERROR: failed to manually attach program prog=__trace_icmp_reply, func=icmp_reply.constprop.0 ERROR: failed to manually attach program prog=__trace_ping_lookup, func=ping_lookup.constprop.0 ERROR: failed to load kprobe-based bpf root@e04-g4-single:/tmp#
没有特殊的内核配置的要求,ubuntu的内核保持默认就行。麻烦运行一下:
nettrace -p icmp --detail --date --debug
可以查看到更加详细的报错信息。
root@e04-g4-single:/tmp# ./nettrace -p icmp --detail --date --debug DEBUG: command: mount | grep debugfs, status:0 DEBUG: command: cat /sys/kernel/debug/tracing/events/skb/kfree_skb/format 2>/dev/null | grep NOT_SPECIFIED, status:1 DEBUG: trace name=tcp_ack_probe, prog=trace_tcp_ack_probe is made invalid DEBUG: trace name=tcp_probe_timer, prog=__trace_tcp_probe_timer is made invalid DEBUG: trace name=tcp_send_probe0, prog=trace_tcp_send_probe0 is made invalid DEBUG: trace name=tcp_transmit_skb, prog=trace_tcp_transmit_skb is made invalid DEBUG: trace name=tcp_retransmit_timer, prog=__trace_tcp_retransmit_timer is made invalid DEBUG: trace name=tcp_sendmsg_locked, prog=trace_tcp_sendmsg_locked is made invalid DEBUG: trace name=tcp_push_pending_frames, prog=_tracetcp_push_pending_frames is made invalid DEBUG: trace name=inet_listen, prog=trace_inet_listen is made invalid DEBUG: trace name=tcp_write_timer_handler, prog=__trace_tcp_write_timer_handler is made invalid DEBUG: trace name=tcp_v4_destroy_sock, prog=trace_tcp_v4_destroy_sock is made invalid DEBUG: trace name=tcp_close, prog=trace_tcp_close is made invalid DEBUG: command: verlte() { [ "$1" = "$2" ] && echo 0 && return; [ "$1" = "$(/bin/echo -e "$1\n$2" | sort -V | head -n1)" ] && echo -1 && return; echo 1; }; [ ! $(verlte "$(uname -r)" "5.4") -eq -1 0 DEBUG: command: verlte() { [ "$1" = "$2" ] && echo 0 && return; [ "$1" = "$(/bin/echo -e "$1\n$2" | sort -V | head -n1)" ] && echo -1 && return; echo 1; }; [ $(verlte "$(uname -r)" "5.16") -eq -1 ]0 DEBUG: begin to resolve kernel symbol... DEBUG: netif_receive_skb_core.constprop.0 is made manual attach DEBUG: netif_receive_skb_core.constprop.0 is made manual attach WARN: kernel function bond_dev_queue_xmit not founded, skipped DEBUG: trace name=bond_dev_queue_xmit, prog=trace_bond_dev_queue_xmit is made invalid WARN: kernel function nft_do_chain not founded, skipped DEBUG: trace name=nft_do_chain, prog=trace_nft_do_chain is made invalid WARN: kernel function nf_nat_manip_pkt not founded, skipped DEBUG: trace name=nf_nat_manip_pkt, prog=trace_nf_nat_manip_pkt is made invalid WARN: kernel function ipt_do_table not founded, skipped DEBUG: trace name=ipt_do_table, prog=trace_ipt_do_table is made invalid WARN: kernel function ipt_do_table not founded, skipped DEBUG: trace name=ipt_do_table, prog=trace_ipt_do_table_legacy is made invalid DEBUG: ip_rcv_core.constprop.0 is made manual attach WARN: kernel function ip6_forward not founded, skipped DEBUG: trace name=ip6_forward, prog=trace_ip6_forward is made invalid WARN: kernel function ip6_rcv_finish not founded, skipped DEBUG: trace name=ip6_rcv_finish, prog=__trace_ip6_rcv_finish is made invalid WARN: kernel function ip6_rcv_core not founded, skipped DEBUG: trace name=ip6_rcv_core, prog=trace_ip6_rcv_core is made invalid WARN: kernel function ipv6_rcv not founded, skipped DEBUG: trace name=ipv6_rcv, prog=trace_ipv6_rcv is made invalid WARN: kernel function ip6_output not founded, skipped DEBUG: trace name=ip6_output, prog=trace_ip6_output is made invalid WARN: kernel function ip6_finish_output not founded, skipped DEBUG: trace name=ip6_finish_output, prog=trace_ip6_finish_output is made invalid WARN: kernel function ip6_finish_output2 not founded, skipped DEBUG: trace name=ip6_finish_output2, prog=__trace_ip6_finish_output2 is made invalid WARN: kernel function ip6_send_skb not founded, skipped DEBUG: trace name=ip6_send_skb, prog=trace_ip6_send_skb is made invalid WARN: kernel function xfrm4_output not founded, skipped DEBUG: trace name=xfrm4_output, prog=trace_xfrm4_output is made invalid WARN: kernel function xfrm_output not founded, skipped DEBUG: trace name=xfrm_output, prog=trace_xfrm_output is made invalid WARN: kernel function xfrm_output2 not founded, skipped DEBUG: trace name=xfrm_output2, prog=trace_xfrm_output2 is made invalid WARN: kernel function xfrm_output_gso not founded, skipped DEBUG: trace name=xfrm_output_gso, prog=trace_xfrm_output_gso is made invalid WARN: kernel function xfrm_output_resume not founded, skipped DEBUG: trace name=xfrm_output_resume, prog=trace_xfrm_output_resume is made invalid WARN: kernel function xfrm4_transport_output not founded, skipped DEBUG: trace name=xfrm4_transport_output, prog=__trace_xfrm4_transport_output is made invalid WARN: kernel function xfrm4_prepare_output not founded, skipped DEBUG: trace name=xfrm4_prepare_output, prog=trace_xfrm4_prepare_output is made invalid WARN: kernel function xfrm4_policy_check not founded, skipped DEBUG: trace name=xfrm4_policy_check, prog=trace_xfrm4_policy_check is made invalid WARN: kernel function xfrm4_rcv not founded, skipped DEBUG: trace name=xfrm4_rcv, prog=trace_xfrm4_rcv is made invalid WARN: kernel function xfrm_input not founded, skipped DEBUG: trace name=xfrm_input, prog=trace_xfrm_input is made invalid WARN: kernel function xfrm4_transport_input not founded, skipped DEBUG: trace name=xfrm4_transport_input, prog=__trace_xfrm4_transport_input is made invalid WARN: kernel function ah_output not founded, skipped DEBUG: trace name=ah_output, prog=trace_ah_output is made invalid WARN: kernel function esp_output not founded, skipped DEBUG: trace name=esp_output, prog=trace_esp_output is made invalid WARN: kernel function esp_output_tail not founded, skipped DEBUG: trace name=esp_output_tail, prog=trace_esp_output_tail is made invalid WARN: kernel function ah_input not founded, skipped DEBUG: trace name=ah_input, prog=trace_ah_input is made invalid WARN: kernel function esp_input not founded, skipped DEBUG: trace name=esp_input, prog=trace_esp_input is made invalid WARN: kernel function ip_route_input_slow not founded, skipped DEBUG: trace name=ip_route_input_slow, prog=trace_ip_route_input_slow is made invalid WARN: kernel function tcp_v6_rcv not founded, skipped DEBUG: trace name=tcp_v6_rcv, prog=trace_tcp_v6_rcv is made invalid WARN: kernel function tcp_v6_send_reset not founded, skipped DEBUG: trace name=tcp_v6_send_reset, prog=trace_tcp_v6_send_reset is made invalid WARN: kernel function tcp_v6_do_rcv not founded, skipped DEBUG: trace name=tcp_v6_do_rcv, prog=trace_tcp_v6_do_rcv is made invalid WARN: kernel function inet6_lookup_listener not founded, skipped DEBUG: trace name=inet6_lookup_listener, prog=trace_inet6_lookup_listener is made invalid WARN: kernel function xfrm4_udp_encap_rcv not founded, skipped DEBUG: trace name=xfrm4_udp_encap_rcv, prog=trace_xfrm4_udp_encap_rcv is made invalid WARN: kernel function xfrm4_rcv_encap not founded, skipped DEBUG: trace name=xfrm4_rcv_encap, prog=trace_xfrm4_rcv_encap is made invalid WARN: kernel function udp_queue_rcv_skb not founded, skipped DEBUG: trace name=udp_queue_rcv_skb, prog=_traceudp_queue_rcv_skb is made invalid DEBUG: icmp_reply.constprop.0 is made manual attach WARN: kernel function icmpv6_rcv not founded, skipped DEBUG: trace name=icmpv6_rcv, prog=trace_icmpv6_rcv is made invalid WARN: kernel function icmpv6_echo_reply not founded, skipped DEBUG: trace name=icmpv6_echo_reply, prog=trace_icmpv6_echo_reply is made invalid WARN: kernel function ping_queue_rcv_skb not founded, skipped DEBUG: trace name=__ping_queue_rcv_skb, prog=trace_ping_queue_rcv_skb is made invalid DEBUG: ping_lookup.constprop.0 is made manual attach DEBUG: finished to resolve kernel symbol DEBUG: kernel feature probe begin DEBUG: kernel feature probe end DEBUG: backup: valid prog for __netif_receive_skb_core.constprop.0 is trace_netif_receive_skb_core_pskb DEBUG: trace name=__netif_receive_skb_core.constprop.0, prog=trace_netif_receive_skb_core is made invalid for: backup following traces are enabled and valid: tracepoint: napi_gro_receive_entry, prog: trace_napi_gro_receive_entry kprobe: dev_gro_receive, prog: trace_dev_gro_receive kprobe: enqueue_to_backlog, prog: trace_enqueue_to_backlog kprobe: netif_receive_generic_xdp, prog: trace_netif_receive_generic_xdp kprobe: xdp_do_generic_redirect, prog: trace_xdp_do_generic_redirect kprobe: netif_receive_skb_core.constprop.0, prog: trace_netif_receive_skb_core_pskb kprobe: dev_queue_xmit, prog: trace_dev_queue_xmit kprobe: dev_hard_start_xmit, prog: trace_dev_hard_start_xmit kprobe: pfifo_enqueue, prog: trace_pfifo_enqueue kprobe: sch_direct_xmit, prog: trace_sch_direct_xmit kprobe: tcf_classify, prog: trace_tcf_classify kprobe: cls_bpf_classify, prog: trace_cls_bpf_classify kprobe: tcf_bpf_act, prog: trace_tcf_bpf_act kprobe: arp_rcv, prog: trace_arp_rcv kprobe: arp_process, prog: __trace_arp_process kprobe: packet_rcv, prog: trace_packet_rcv kprobe: tpacket_rcv, prog: trace_tpacket_rcv kprobe: packet_direct_xmit, prog: trace_packet_direct_xmit kprobe: nf_hook_slow, prog: trace_nf_hook_slow kprobe: ip_rcv, prog: trace_ip_rcv kprobe: ip_rcv_core.constprop.0, prog: trace_ip_rcv_core kprobe: ip_rcv_finish, prog: __trace_ip_rcv_finish kprobe: ip_local_deliver, prog: trace_ip_local_deliver kprobe: ip_local_deliver_finish, prog: trace_ip_local_deliver_finish kprobe: ip_forward, prog: trace_ip_forward kprobe: ip_forward_finish, prog: trace_ip_forward_finish kprobe: ip_output, prog: trace_ip_output kprobe: ip_finish_output, prog: trace_ip_finish_output kprobe: ip_finish_output_gso, prog: trace_ip_finish_output_gso kprobe: ip_finish_output2, prog: trace_ip_finish_output2 kprobe: ip6_local_out, prog: trace_ip6_local_out kprobe: fib_validate_source, prog: trace_fib_validate_source kprobe: tcp_v4_rcv, prog: trace_tcp_v4_rcv kprobe: tcp_filter, prog: trace_tcp_filter kprobe: tcp_child_process, prog: trace_tcp_child_process kprobe: tcp_v4_send_reset, prog: trace_tcp_v4_send_reset kprobe: tcp_v4_do_rcv, prog: trace_tcp_v4_do_rcv kprobe: tcp_rcv_established, prog: trace_tcp_rcv_established kprobe: tcp_rcv_state_process, prog: trace_tcp_rcv_state_process kprobe: tcp_queue_rcv, prog: trace_tcp_queue_rcv kprobe: tcp_ack, prog: trace_tcp_ack kprobe: inet_lookup_listener, prog: trace_inet_lookup_listener kprobe: udp_rcv, prog: trace_udp_rcv kprobe: udp_unicast_rcv_skb, prog: trace_udp_unicast_rcv_skb kprobe: udp_queue_rcv_skb, prog: trace_udp_queue_rcv_skb kprobe: udp_enqueue_schedule_skb, prog: trace_udp_enqueue_schedule_skb kprobe: icmp_rcv, prog: __trace_icmp_rcv kprobe: icmp_echo, prog: trace_icmp_echo kprobe: icmp_reply.constprop.0, prog: trace_icmp_reply kprobe: ping_rcv, prog: __trace_ping_rcv kprobe: ping_queue_rcv_skb, prog: trace_ping_queue_rcv_skb kprobe: ping_lookup.constprop.0, prog: trace_ping_lookup kprobe: kfree_skb, prog: trace_kfree_skb kprobe/kretprobe: skb_clone, prog: trace_skb_clone tracepoint: consume_skb, prog: trace_consume_skb tracepoint: kfree_skb, prog: trace_kfree_skb DEBUG: eBPF is opened successfully DEBUG: prog: trace_dev_gro_receive is made no-autoload DEBUG: prog: trace_enqueue_to_backlog is made no-autoload DEBUG: prog: trace_netif_receive_generic_xdp is made no-autoload DEBUG: prog: trace_xdp_do_generic_redirect is made no-autoload DEBUG: prog: _tracenetif_receive_skb_core is made no-autoload DEBUG: prog: trace___netif_receive_skb_core is made no-autoload DEBUG: prog: trace_netif_receive_skb_core_pskb is made no-autoload DEBUG: prog: _tracedev_queue_xmit is made no-autoload DEBUG: prog: trace_dev_hard_start_xmit is made no-autoload DEBUG: prog: trace_pfifo_enqueue is made no-autoload DEBUG: prog: trace_sch_direct_xmit is made no-autoload DEBUG: prog: trace_tcf_classify is made no-autoload DEBUG: prog: trace_cls_bpf_classify is made no-autoload DEBUG: prog: trace_tcf_bpf_act is made no-autoload DEBUG: prog: trace_ipvlan_queue_xmit is made no-autoload DEBUG: prog: __trace_ipvlan_queue_xmit is made no-autoload DEBUG: prog: trace_ipvlan_handle_frame is made no-autoload DEBUG: prog: trace_ipvlan_handle_frame is made no-autoload DEBUG: prog: __trace_ipvlan_rcv_frame is made no-autoload DEBUG: prog: trace_ipvlan_rcv_frame is made no-autoload DEBUG: prog: trace_ipvlan_xmit_mode_l3 is made no-autoload DEBUG: prog: trace_ipvlan_xmit_mode_l3 is made no-autoload DEBUG: prog: trace_ipvlan_process_v4_outbound is made no-autoload DEBUG: prog: trace_ipvlan_process_v4_outbound is made no-autoload DEBUG: prog: trace_br_nf_pre_routing is made no-autoload DEBUG: prog: trace_br_nf_pre_routing is made no-autoload DEBUG: prog: trace_br_nf_forward_ip is made no-autoload DEBUG: prog: trace_br_nf_forward_ip is made no-autoload DEBUG: prog: trace_br_nf_forward_arp is made no-autoload DEBUG: prog: trace_br_nf_forward_arp is made no-autoload DEBUG: prog: trace_br_nf_post_routing is made no-autoload DEBUG: prog: trace_br_nf_post_routing is made no-autoload DEBUG: prog: trace_arp_rcv is made no-autoload DEBUG: prog: trace_arp_process is made no-autoload DEBUG: prog: trace_bond_dev_queue_xmit is made no-autoload DEBUG: prog: trace_bond_dev_queue_xmit is made no-autoload DEBUG: prog: trace_packet_rcv is made no-autoload DEBUG: prog: trace_tpacket_rcv is made no-autoload DEBUG: prog: trace_packet_direct_xmit is made no-autoload DEBUG: prog: __trace_nft_do_chain is made no-autoload DEBUG: prog: trace_nft_do_chain is made no-autoload DEBUG: prog: trace_nf_nat_manip_pkt is made no-autoload DEBUG: prog: trace_nf_nat_manip_pkt is made no-autoload DEBUG: prog: trace_nf_hook_slow is made no-autoload DEBUG: prog: __trace_ipt_do_table is made no-autoload DEBUG: prog: trace_ipt_do_table is made no-autoload DEBUG: prog: trace_ipt_do_table_legacy is made no-autoload DEBUG: prog: trace_ipt_do_table_legacy is made no-autoload DEBUG: prog: trace_ipv4_confirm is made no-autoload DEBUG: prog: trace_ipv4_confirm is made no-autoload DEBUG: prog: trace_nf_confirm is made no-autoload DEBUG: prog: trace_nf_confirm is made no-autoload DEBUG: prog: trace_ipv4_conntrack_in is made no-autoload DEBUG: prog: __trace_ipv4_conntrack_in is made no-autoload DEBUG: prog: trace_nf_conntrack_in is made no-autoload DEBUG: prog: trace_nf_conntrack_in is made no-autoload DEBUG: prog: trace_ipv4_pkt_to_tuple is made no-autoload DEBUG: prog: trace_ipv4_pkt_to_tuple is made no-autoload DEBUG: prog: trace_tcp_new is made no-autoload DEBUG: prog: trace_tcp_new is made no-autoload DEBUG: prog: trace_tcp_pkt_to_tuple is made no-autoload DEBUG: prog: trace_tcp_pkt_to_tuple is made no-autoload DEBUG: prog: __trace_resolve_normal_ct is made no-autoload DEBUG: prog: trace_resolve_normal_ct is made no-autoload DEBUG: prog: trace_tcp_packet is made no-autoload DEBUG: prog: trace_tcp_packet is made no-autoload DEBUG: prog: trace_tcp_in_window is made no-autoload DEBUG: prog: __trace_tcp_in_window is made no-autoload DEBUG: prog: trace_nf_ct_refresh_acct is made no-autoload DEBUG: prog: trace_nf_ct_refresh_acct is made no-autoload DEBUG: prog: trace_ip_rcv is made no-autoload DEBUG: prog: trace_ip_rcv_core is made no-autoload DEBUG: prog: __trace_ip_rcv_finish is made no-autoload DEBUG: prog: trace_ip_local_deliver is made no-autoload DEBUG: prog: trace_ip_local_deliver_finish is made no-autoload DEBUG: prog: trace_ip_forward is made no-autoload DEBUG: prog: trace_ip_forward_finish is made no-autoload DEBUG: prog: trace_ip6_forward is made no-autoload DEBUG: prog: trace_ip6_forward is made no-autoload DEBUG: prog: __trace_ip6_rcv_finish is made no-autoload DEBUG: prog: trace_ip6_rcv_finish is made no-autoload DEBUG: prog: trace_ip6_rcv_core is made no-autoload DEBUG: prog: __trace_ip6_rcv_core is made no-autoload DEBUG: prog: trace_ipv6_rcv is made no-autoload DEBUG: prog: trace_ipv6_rcv is made no-autoload DEBUG: prog: trace_ip_output is made no-autoload DEBUG: prog: trace_ip_finish_output is made no-autoload DEBUG: prog: trace_ip_finish_output_gso is made no-autoload DEBUG: prog: trace_ip_finish_output2 is made no-autoload DEBUG: prog: trace_ip6_output is made no-autoload DEBUG: prog: trace_ip6_output is made no-autoload DEBUG: prog: trace_ip6_finish_output is made no-autoload DEBUG: prog: trace_ip6_finish_output is made no-autoload DEBUG: prog: __trace_ip6_finish_output2 is made no-autoload DEBUG: prog: trace_ip6_finish_output2 is made no-autoload DEBUG: prog: trace_ip6_send_skb is made no-autoload DEBUG: prog: __trace_ip6_send_skb is made no-autoload DEBUG: prog: trace_ip6_local_out is made no-autoload DEBUG: prog: trace_xfrm4_output is made no-autoload DEBUG: prog: trace_xfrm4_output is made no-autoload DEBUG: prog: trace_xfrm_output is made no-autoload DEBUG: prog: trace_xfrm_output is made no-autoload DEBUG: prog: trace_xfrm_output2 is made no-autoload DEBUG: prog: trace_xfrm_output2 is made no-autoload DEBUG: prog: trace_xfrm_output_gso is made no-autoload DEBUG: prog: __trace_xfrm_output_gso is made no-autoload DEBUG: prog: trace_xfrm_output_resume is made no-autoload DEBUG: prog: trace_xfrm_output_resume is made no-autoload DEBUG: prog: __trace_xfrm4_transport_output is made no-autoload DEBUG: prog: trace_xfrm4_transport_output is made no-autoload DEBUG: prog: trace_xfrm4_prepare_output is made no-autoload DEBUG: prog: __trace_xfrm4_prepare_output is made no-autoload DEBUG: prog: trace_xfrm4_policy_check is made no-autoload DEBUG: prog: trace_xfrm4_policy_check is made no-autoload DEBUG: prog: trace_xfrm4_rcv is made no-autoload DEBUG: prog: trace_xfrm4_rcv is made no-autoload DEBUG: prog: trace_xfrm_input is made no-autoload DEBUG: prog: trace_xfrm_input is made no-autoload DEBUG: prog: __trace_xfrm4_transport_input is made no-autoload DEBUG: prog: trace_xfrm4_transport_input is made no-autoload DEBUG: prog: trace_ah_output is made no-autoload DEBUG: prog: trace_ah_output is made no-autoload DEBUG: prog: trace_esp_output is made no-autoload DEBUG: prog: trace_esp_output is made no-autoload DEBUG: prog: trace_esp_output_tail is made no-autoload DEBUG: prog: __trace_esp_output_tail is made no-autoload DEBUG: prog: trace_ah_input is made no-autoload DEBUG: prog: trace_ah_input is made no-autoload DEBUG: prog: trace_esp_input is made no-autoload DEBUG: prog: trace_esp_input is made no-autoload DEBUG: prog: trace_fib_validate_source is made no-autoload DEBUG: prog: trace_ip_route_input_slow is made no-autoload DEBUG: prog: trace_ip_route_input_slow is made no-autoload DEBUG: prog: trace_tcp_v4_rcv is made no-autoload DEBUG: prog: __trace_tcp_v6_rcv is made no-autoload DEBUG: prog: trace_tcp_v6_rcv is made no-autoload DEBUG: prog: trace_tcp_filter is made no-autoload DEBUG: prog: trace_tcp_child_process is made no-autoload DEBUG: prog: trace_tcp_v4_send_reset is made no-autoload DEBUG: prog: trace_tcp_v6_send_reset is made no-autoload DEBUG: prog: trace_tcp_v6_send_reset is made no-autoload DEBUG: prog: trace_tcp_v4_do_rcv is made no-autoload DEBUG: prog: trace_tcp_v6_do_rcv is made no-autoload DEBUG: prog: trace_tcp_v6_do_rcv is made no-autoload DEBUG: prog: trace_tcp_rcv_established is made no-autoload DEBUG: prog: trace_tcp_rcv_state_process is made no-autoload DEBUG: prog: trace_tcp_queue_rcv is made no-autoload DEBUG: prog: trace_tcp_ack_probe is made no-autoload DEBUG: prog: trace_tcp_ack_probe is made no-autoload DEBUG: prog: trace_tcp_ack is made no-autoload DEBUG: prog: trace_tcp_probe_timer is made no-autoload DEBUG: prog: trace_tcp_probe_timer is made no-autoload DEBUG: prog: trace_tcp_send_probe0 is made no-autoload DEBUG: prog: __trace_tcp_send_probe0 is made no-autoload DEBUG: prog: trace_inet_lookup_listener is made no-autoload DEBUG: prog: __trace_inet6_lookup_listener is made no-autoload DEBUG: prog: trace_inet6_lookup_listener is made no-autoload DEBUG: prog: trace_tcp_transmit_skb is made no-autoload DEBUG: prog: trace_tcp_transmit_skb is made no-autoload DEBUG: prog: trace_tcp_retransmit_timer is made no-autoload DEBUG: prog: __trace_tcp_retransmit_timer is made no-autoload DEBUG: prog: trace_tcp_sendmsg_locked is made no-autoload DEBUG: prog: trace_tcp_sendmsg_locked is made no-autoload DEBUG: prog: _tracetcp_push_pending_frames is made no-autoload DEBUG: prog: trace___tcp_push_pending_frames is made no-autoload DEBUG: prog: trace_inet_listen is made no-autoload DEBUG: prog: trace_inet_listen is made no-autoload DEBUG: prog: trace_tcp_write_timer_handler is made no-autoload DEBUG: prog: trace_tcp_write_timer_handler is made no-autoload DEBUG: prog: trace_tcp_v4_destroy_sock is made no-autoload DEBUG: prog: trace_tcp_v4_destroy_sock is made no-autoload DEBUG: prog: trace_tcp_close is made no-autoload DEBUG: prog: trace_tcp_close is made no-autoload DEBUG: prog: trace_udp_rcv is made no-autoload DEBUG: prog: trace_udp_unicast_rcv_skb is made no-autoload DEBUG: prog: trace_udp_queue_rcv_skb is made no-autoload DEBUG: prog: trace_xfrm4_udp_encap_rcv is made no-autoload DEBUG: prog: trace_xfrm4_udp_encap_rcv is made no-autoload DEBUG: prog: trace_xfrm4_rcv_encap is made no-autoload DEBUG: prog: __trace_xfrm4_rcv_encap is made no-autoload DEBUG: prog: trace_udp_queue_rcv_skb is made no-autoload DEBUG: prog: trace_udp_queue_rcv_skb is made no-autoload DEBUG: prog: trace_udp_enqueue_schedule_skb is made no-autoload DEBUG: prog: trace_icmp_rcv is made no-autoload DEBUG: prog: trace_icmp_echo is made no-autoload DEBUG: prog: trace_icmp_reply is made no-autoload DEBUG: prog: trace_icmpv6_rcv is made no-autoload DEBUG: prog: trace_icmpv6_rcv is made no-autoload DEBUG: prog: trace_icmpv6_echo_reply is made no-autoload DEBUG: prog: __trace_icmpv6_echo_reply is made no-autoload DEBUG: prog: trace_ping_rcv is made no-autoload DEBUG: prog: trace___ping_queue_rcv_skb is made no-autoload DEBUG: prog: trace_ping_queue_rcv_skb is made no-autoload DEBUG: prog: trace_ping_queue_rcv_skb is made no-autoload DEBUG: prog: trace_ping_lookup is made no-autoload DEBUG: prog: trace_kfree_skb is made no-autoload DEBUG: eBPF is loaded successfully DEBUG: begin to attach eBPF program... DEBUG: command: (echo 'p:netif_receive_skb_core_constprop_0 netif_receive_skb_core.constprop.0' >> /sys/kernel/debug/tracing/kprobe_events) 2>&1, status:1 WARN: failed to create kprobe: netif_receive_skb_core_constprop_0 ERROR: failed to manually attach program prog=trace___netif_receive_skb_core, func=netif_receive_skb_core.constprop.0 DEBUG: command: (echo 'p:netif_receive_skb_core_constprop_0 netif_receive_skb_core.constprop.0' >> /sys/kernel/debug/tracing/kprobe_events) 2>&1, status:1 WARN: failed to create kprobe: netif_receive_skb_core_constprop_0 ERROR: failed to manually attach program prog=trace_netif_receive_skb_core_pskb, func=netif_receive_skb_core.constprop.0 DEBUG: command: (echo 'p:ip_rcv_core_constprop_0 ip_rcv_core.constprop.0' >> /sys/kernel/debug/tracing/kprobe_events) 2>&1, status:1 WARN: failed to create kprobe: ip_rcv_core_constprop_0 ERROR: failed to manually attach program prog=trace_ip_rcv_core, func=ip_rcv_core.constprop.0 DEBUG: command: (echo 'p:icmp_reply_constprop_0 icmp_reply.constprop.0' >> /sys/kernel/debug/tracing/kprobe_events) 2>&1, status:1 WARN: failed to create kprobe: icmp_reply_constprop_0 ERROR: failed to manually attach program prog=trace_icmp_reply, func=icmp_reply.constprop.0 DEBUG: command: (echo 'p:ping_lookup_constprop_0 ping_lookup.constprop.0' >> /sys/kernel/debug/tracing/kprobe_events) 2>&1, status:1 WARN: failed to create kprobe: ping_lookup_constprop_0 ERROR: failed to manually attach program prog=trace_ping_lookup, func=ping_lookup.constprop.0 libbpf: failed to add legacy kprobe event for 'dev_gro_receive+0x0': No such file or directory libbpf: prog '__trace_dev_gro_receive': failed to create kprobe 'dev_gro_receive+0x0' perf event: No such file or directory libbpf: prog 'trace_dev_gro_receive': failed to auto-attach: -2 ERROR: failed to load kprobe-based bpf
补充一下,我这个是aarch64 编译的,在arm开发板上运行的结果
开发板的内核版本5.10.110
内核是自己编译的吗?感觉像是内核的一些配置没有打开。请确保以下内核配置处于开启状态:
CONFIG_KPROBE_EVENTS CONFIG_BPF_SYSCALL CONFIG_FTRACE CONFIG_DYNAMIC_FTRACE
是的 内核是自己编译的。我按照bcc的kernel config, 应该是打开了如下内核配置 CONFIG_BPF=y CONFIG_BPF_SYSCALL=y CONFIG_NET_CLS_BPF=y CONFIG_NET_ACT_BPF=y CONFIG_BPF_JIT=y CONFIG_HAVE_EBPF_JIT=y CONFIG_BPF_EVENTS=y CONFIG_IKHEADERS=y
CONFIG_NET_SCH_SFQ=y CONFIG_NET_ACT_POLICE=y CONFIG_NET_ACT_GACT=y
不确定是否还需要其他的
内核是自己编译的吗?感觉像是内核的一些配置没有打开。请确保以下内核配置处于开启状态:
CONFIG_KPROBE_EVENTS CONFIG_BPF_SYSCALL CONFIG_FTRACE CONFIG_DYNAMIC_FTRACE
加上这些配置,刚才那个失败的错误消失了,但是好像追踪没有输出 root@e04-g4-single:/tmp# ./nettrace -p icmp --detail --date --debug DEBUG: command: mount | grep debugfs, status:0 DEBUG: command: cat /sys/kernel/debug/tracing/events/skb/kfree_skb/format 2>/dev/null | grep NOT_SPECIFIED, status:1 DEBUG: trace name=tcp_ack_probe, prog=trace_tcp_ack_probe is made invalid DEBUG: trace name=tcp_probe_timer, prog=__trace_tcp_probe_timer is made invalid DEBUG: trace name=tcp_send_probe0, prog=trace_tcp_send_probe0 is made invalid DEBUG: trace name=tcp_transmit_skb, prog=trace_tcp_transmit_skb is made invalid DEBUG: trace name=tcp_retransmit_timer, prog=__trace_tcp_retransmit_timer is made invalid DEBUG: trace name=tcp_sendmsg_locked, prog=trace_tcp_sendmsg_locked is made invalid DEBUG: trace name=tcp_push_pending_frames, prog=_tracetcp_push_pending_frames is made invalid DEBUG: trace name=inet_listen, prog=trace_inet_listen is made invalid DEBUG: trace name=tcp_write_timer_handler, prog=__trace_tcp_write_timer_handler is made invalid DEBUG: trace name=tcp_v4_destroy_sock, prog=trace_tcp_v4_destroy_sock is made invalid DEBUG: trace name=tcp_close, prog=trace_tcp_close is made invalid DEBUG: command: verlte() { [ "$1" = "$2" ] && echo 0 && return; [ "$1" = "$(/bin/echo -e "$1\n$2" | sort -V | head -n1)" ] && echo -1 && return; echo 1; }; [ ! $(verlte "$(uname -r)" "5.4") -eq -1 ], status:0 DEBUG: command: verlte() { [ "$1" = "$2" ] && echo 0 && return; [ "$1" = "$(/bin/echo -e "$1\n$2" | sort -V | head -n1)" ] && echo -1 && return; echo 1; }; [ $(verlte "$(uname -r)" "5.16") -eq -1 ], status:0 DEBUG: begin to resolve kernel symbol... DEBUG: netif_receive_skb_core.constprop.0 is made manual attach DEBUG: netif_receive_skb_core.constprop.0 is made manual attach WARN: kernel function bond_dev_queue_xmit not founded, skipped DEBUG: trace name=bond_dev_queue_xmit, prog=trace_bond_dev_queue_xmit is made invalid WARN: kernel function nft_do_chain not founded, skipped DEBUG: trace name=nft_do_chain, prog=trace_nft_do_chain is made invalid WARN: kernel function nf_nat_manip_pkt not founded, skipped DEBUG: trace name=nf_nat_manip_pkt, prog=trace_nf_nat_manip_pkt is made invalid WARN: kernel function ipt_do_table not founded, skipped DEBUG: trace name=ipt_do_table, prog=trace_ipt_do_table is made invalid WARN: kernel function ipt_do_table not founded, skipped DEBUG: trace name=ipt_do_table, prog=trace_ipt_do_table_legacy is made invalid DEBUG: ip_rcv_core.constprop.0 is made manual attach WARN: kernel function ip6_forward not founded, skipped DEBUG: trace name=ip6_forward, prog=trace_ip6_forward is made invalid WARN: kernel function ip6_rcv_finish not founded, skipped DEBUG: trace name=ip6_rcv_finish, prog=__trace_ip6_rcv_finish is made invalid WARN: kernel function ip6_rcv_core not founded, skipped DEBUG: trace name=ip6_rcv_core, prog=trace_ip6_rcv_core is made invalid WARN: kernel function ipv6_rcv not founded, skipped DEBUG: trace name=ipv6_rcv, prog=trace_ipv6_rcv is made invalid WARN: kernel function ip6_output not founded, skipped DEBUG: trace name=ip6_output, prog=trace_ip6_output is made invalid WARN: kernel function ip6_finish_output not founded, skipped DEBUG: trace name=ip6_finish_output, prog=trace_ip6_finish_output is made invalid WARN: kernel function ip6_finish_output2 not founded, skipped DEBUG: trace name=ip6_finish_output2, prog=__trace_ip6_finish_output2 is made invalid WARN: kernel function ip6_send_skb not founded, skipped DEBUG: trace name=ip6_send_skb, prog=trace_ip6_send_skb is made invalid WARN: kernel function xfrm4_output not founded, skipped DEBUG: trace name=xfrm4_output, prog=trace_xfrm4_output is made invalid WARN: kernel function xfrm_output not founded, skipped DEBUG: trace name=xfrm_output, prog=trace_xfrm_output is made invalid WARN: kernel function xfrm_output2 not founded, skipped DEBUG: trace name=xfrm_output2, prog=trace_xfrm_output2 is made invalid WARN: kernel function xfrm_output_gso not founded, skipped DEBUG: trace name=xfrm_output_gso, prog=trace_xfrm_output_gso is made invalid WARN: kernel function xfrm_output_resume not founded, skipped DEBUG: trace name=xfrm_output_resume, prog=trace_xfrm_output_resume is made invalid WARN: kernel function xfrm4_transport_output not founded, skipped DEBUG: trace name=xfrm4_transport_output, prog=__trace_xfrm4_transport_output is made invalid WARN: kernel function xfrm4_prepare_output not founded, skipped DEBUG: trace name=xfrm4_prepare_output, prog=trace_xfrm4_prepare_output is made invalid WARN: kernel function xfrm4_policy_check not founded, skipped DEBUG: trace name=xfrm4_policy_check, prog=trace_xfrm4_policy_check is made invalid WARN: kernel function xfrm4_rcv not founded, skipped DEBUG: trace name=xfrm4_rcv, prog=trace_xfrm4_rcv is made invalid WARN: kernel function xfrm_input not founded, skipped DEBUG: trace name=xfrm_input, prog=trace_xfrm_input is made invalid WARN: kernel function xfrm4_transport_input not founded, skipped DEBUG: trace name=xfrm4_transport_input, prog=__trace_xfrm4_transport_input is made invalid WARN: kernel function ah_output not founded, skipped DEBUG: trace name=ah_output, prog=trace_ah_output is made invalid WARN: kernel function esp_output not founded, skipped DEBUG: trace name=esp_output, prog=trace_esp_output is made invalid WARN: kernel function esp_output_tail not founded, skipped DEBUG: trace name=esp_output_tail, prog=trace_esp_output_tail is made invalid WARN: kernel function ah_input not founded, skipped DEBUG: trace name=ah_input, prog=trace_ah_input is made invalid WARN: kernel function esp_input not founded, skipped DEBUG: trace name=esp_input, prog=trace_esp_input is made invalid WARN: kernel function ip_route_input_slow not founded, skipped DEBUG: trace name=ip_route_input_slow, prog=trace_ip_route_input_slow is made invalid WARN: kernel function tcp_v6_rcv not founded, skipped DEBUG: trace name=tcp_v6_rcv, prog=trace_tcp_v6_rcv is made invalid WARN: kernel function tcp_v6_send_reset not founded, skipped DEBUG: trace name=tcp_v6_send_reset, prog=trace_tcp_v6_send_reset is made invalid WARN: kernel function tcp_v6_do_rcv not founded, skipped DEBUG: trace name=tcp_v6_do_rcv, prog=trace_tcp_v6_do_rcv is made invalid WARN: kernel function inet6_lookup_listener not founded, skipped DEBUG: trace name=inet6_lookup_listener, prog=trace_inet6_lookup_listener is made invalid WARN: kernel function xfrm4_udp_encap_rcv not founded, skipped DEBUG: trace name=xfrm4_udp_encap_rcv, prog=trace_xfrm4_udp_encap_rcv is made invalid WARN: kernel function xfrm4_rcv_encap not founded, skipped DEBUG: trace name=xfrm4_rcv_encap, prog=trace_xfrm4_rcv_encap is made invalid WARN: kernel function udp_queue_rcv_skb not founded, skipped DEBUG: trace name=udp_queue_rcv_skb, prog=_traceudp_queue_rcv_skb is made invalid DEBUG: icmp_reply.constprop.0 is made manual attach WARN: kernel function icmpv6_rcv not founded, skipped DEBUG: trace name=icmpv6_rcv, prog=trace_icmpv6_rcv is made invalid WARN: kernel function icmpv6_echo_reply not founded, skipped DEBUG: trace name=icmpv6_echo_reply, prog=trace_icmpv6_echo_reply is made invalid WARN: kernel function ping_queue_rcv_skb not founded, skipped DEBUG: trace name=__ping_queue_rcv_skb, prog=trace_ping_queue_rcv_skb is made invalid DEBUG: ping_lookup.constprop.0 is made manual attach DEBUG: finished to resolve kernel symbol DEBUG: kernel feature probe begin DEBUG: kernel feature probe end DEBUG: backup: valid prog for __netif_receive_skb_core.constprop.0 is trace_netif_receive_skb_core_pskb DEBUG: trace name=__netif_receive_skb_core.constprop.0, prog=trace_netif_receive_skb_core is made invalid for: backup following traces are enabled and valid: tracepoint: napi_gro_receive_entry, prog: trace_napi_gro_receive_entry kprobe: dev_gro_receive, prog: trace_dev_gro_receive kprobe: enqueue_to_backlog, prog: trace_enqueue_to_backlog kprobe: netif_receive_generic_xdp, prog: trace_netif_receive_generic_xdp kprobe: xdp_do_generic_redirect, prog: trace_xdp_do_generic_redirect kprobe: netif_receive_skb_core.constprop.0, prog: trace_netif_receive_skb_core_pskb kprobe: dev_queue_xmit, prog: trace_dev_queue_xmit kprobe: dev_hard_start_xmit, prog: trace_dev_hard_start_xmit kprobe: pfifo_enqueue, prog: trace_pfifo_enqueue kprobe: sch_direct_xmit, prog: trace_sch_direct_xmit kprobe: tcf_classify, prog: trace_tcf_classify kprobe: cls_bpf_classify, prog: trace_cls_bpf_classify kprobe: tcf_bpf_act, prog: trace_tcf_bpf_act kprobe: arp_rcv, prog: trace_arp_rcv kprobe: arp_process, prog: __trace_arp_process kprobe: packet_rcv, prog: trace_packet_rcv kprobe: tpacket_rcv, prog: trace_tpacket_rcv kprobe: packet_direct_xmit, prog: trace_packet_direct_xmit kprobe: nf_hook_slow, prog: trace_nf_hook_slow kprobe: ip_rcv, prog: trace_ip_rcv kprobe: ip_rcv_core.constprop.0, prog: trace_ip_rcv_core kprobe: ip_rcv_finish, prog: __trace_ip_rcv_finish kprobe: ip_local_deliver, prog: trace_ip_local_deliver kprobe: ip_local_deliver_finish, prog: trace_ip_local_deliver_finish kprobe: ip_forward, prog: trace_ip_forward kprobe: ip_forward_finish, prog: trace_ip_forward_finish kprobe: ip_output, prog: trace_ip_output kprobe: ip_finish_output, prog: trace_ip_finish_output kprobe: ip_finish_output_gso, prog: trace_ip_finish_output_gso kprobe: ip_finish_output2, prog: trace_ip_finish_output2 kprobe: ip6_local_out, prog: trace_ip6_local_out kprobe: fib_validate_source, prog: trace_fib_validate_source kprobe: tcp_v4_rcv, prog: trace_tcp_v4_rcv kprobe: tcp_filter, prog: trace_tcp_filter kprobe: tcp_child_process, prog: trace_tcp_child_process kprobe: tcp_v4_send_reset, prog: trace_tcp_v4_send_reset kprobe: tcp_v4_do_rcv, prog: trace_tcp_v4_do_rcv kprobe: tcp_rcv_established, prog: trace_tcp_rcv_established kprobe: tcp_rcv_state_process, prog: trace_tcp_rcv_state_process kprobe: tcp_queue_rcv, prog: trace_tcp_queue_rcv kprobe: tcp_ack, prog: trace_tcp_ack kprobe: inet_lookup_listener, prog: trace_inet_lookup_listener kprobe: udp_rcv, prog: trace_udp_rcv kprobe: udp_unicast_rcv_skb, prog: trace_udp_unicast_rcv_skb kprobe: udp_queue_rcv_skb, prog: trace_udp_queue_rcv_skb kprobe: udp_enqueue_schedule_skb, prog: trace_udp_enqueue_schedule_skb kprobe: icmp_rcv, prog: __trace_icmp_rcv kprobe: icmp_echo, prog: trace_icmp_echo kprobe: icmp_reply.constprop.0, prog: trace_icmp_reply kprobe: ping_rcv, prog: __trace_ping_rcv kprobe: ping_queue_rcv_skb, prog: trace_ping_queue_rcv_skb kprobe: ping_lookup.constprop.0, prog: trace_ping_lookup kprobe: kfree_skb, prog: trace_kfree_skb kprobe/kretprobe: skb_clone, prog: trace_skb_clone tracepoint: consume_skb, prog: trace_consume_skb tracepoint: kfree_skb, prog: trace_kfree_skb DEBUG: eBPF is opened successfully DEBUG: prog: trace_dev_gro_receive is made no-autoload DEBUG: prog: trace_enqueue_to_backlog is made no-autoload DEBUG: prog: trace_netif_receive_generic_xdp is made no-autoload DEBUG: prog: trace_xdp_do_generic_redirect is made no-autoload DEBUG: prog: _tracenetif_receive_skb_core is made no-autoload DEBUG: prog: trace___netif_receive_skb_core is made no-autoload DEBUG: prog: trace_netif_receive_skb_core_pskb is made no-autoload DEBUG: prog: _tracedev_queue_xmit is made no-autoload DEBUG: prog: trace_dev_hard_start_xmit is made no-autoload DEBUG: prog: trace_pfifo_enqueue is made no-autoload DEBUG: prog: trace_sch_direct_xmit is made no-autoload DEBUG: prog: trace_tcf_classify is made no-autoload DEBUG: prog: trace_cls_bpf_classify is made no-autoload DEBUG: prog: trace_tcf_bpf_act is made no-autoload DEBUG: prog: trace_ipvlan_queue_xmit is made no-autoload DEBUG: prog: __trace_ipvlan_queue_xmit is made no-autoload DEBUG: prog: trace_ipvlan_handle_frame is made no-autoload DEBUG: prog: trace_ipvlan_handle_frame is made no-autoload DEBUG: prog: __trace_ipvlan_rcv_frame is made no-autoload DEBUG: prog: trace_ipvlan_rcv_frame is made no-autoload DEBUG: prog: trace_ipvlan_xmit_mode_l3 is made no-autoload DEBUG: prog: trace_ipvlan_xmit_mode_l3 is made no-autoload DEBUG: prog: trace_ipvlan_process_v4_outbound is made no-autoload DEBUG: prog: trace_ipvlan_process_v4_outbound is made no-autoload DEBUG: prog: trace_br_nf_pre_routing is made no-autoload DEBUG: prog: trace_br_nf_pre_routing is made no-autoload DEBUG: prog: trace_br_nf_forward_ip is made no-autoload DEBUG: prog: trace_br_nf_forward_ip is made no-autoload DEBUG: prog: trace_br_nf_forward_arp is made no-autoload DEBUG: prog: trace_br_nf_forward_arp is made no-autoload DEBUG: prog: trace_br_nf_post_routing is made no-autoload DEBUG: prog: trace_br_nf_post_routing is made no-autoload DEBUG: prog: trace_arp_rcv is made no-autoload DEBUG: prog: trace_arp_process is made no-autoload DEBUG: prog: trace_bond_dev_queue_xmit is made no-autoload DEBUG: prog: trace_bond_dev_queue_xmit is made no-autoload DEBUG: prog: trace_packet_rcv is made no-autoload DEBUG: prog: trace_tpacket_rcv is made no-autoload DEBUG: prog: trace_packet_direct_xmit is made no-autoload DEBUG: prog: __trace_nft_do_chain is made no-autoload DEBUG: prog: trace_nft_do_chain is made no-autoload DEBUG: prog: trace_nf_nat_manip_pkt is made no-autoload DEBUG: prog: trace_nf_nat_manip_pkt is made no-autoload DEBUG: prog: trace_nf_hook_slow is made no-autoload DEBUG: prog: __trace_ipt_do_table is made no-autoload DEBUG: prog: trace_ipt_do_table is made no-autoload DEBUG: prog: trace_ipt_do_table_legacy is made no-autoload DEBUG: prog: trace_ipt_do_table_legacy is made no-autoload DEBUG: prog: trace_ipv4_confirm is made no-autoload DEBUG: prog: trace_ipv4_confirm is made no-autoload DEBUG: prog: trace_nf_confirm is made no-autoload DEBUG: prog: trace_nf_confirm is made no-autoload DEBUG: prog: trace_ipv4_conntrack_in is made no-autoload DEBUG: prog: __trace_ipv4_conntrack_in is made no-autoload DEBUG: prog: trace_nf_conntrack_in is made no-autoload DEBUG: prog: trace_nf_conntrack_in is made no-autoload DEBUG: prog: trace_ipv4_pkt_to_tuple is made no-autoload DEBUG: prog: trace_ipv4_pkt_to_tuple is made no-autoload DEBUG: prog: trace_tcp_new is made no-autoload DEBUG: prog: trace_tcp_new is made no-autoload DEBUG: prog: trace_tcp_pkt_to_tuple is made no-autoload DEBUG: prog: trace_tcp_pkt_to_tuple is made no-autoload DEBUG: prog: __trace_resolve_normal_ct is made no-autoload DEBUG: prog: trace_resolve_normal_ct is made no-autoload DEBUG: prog: trace_tcp_packet is made no-autoload DEBUG: prog: trace_tcp_packet is made no-autoload DEBUG: prog: trace_tcp_in_window is made no-autoload DEBUG: prog: __trace_tcp_in_window is made no-autoload DEBUG: prog: trace_nf_ct_refresh_acct is made no-autoload DEBUG: prog: trace_nf_ct_refresh_acct is made no-autoload DEBUG: prog: trace_ip_rcv is made no-autoload DEBUG: prog: trace_ip_rcv_core is made no-autoload DEBUG: prog: __trace_ip_rcv_finish is made no-autoload DEBUG: prog: trace_ip_local_deliver is made no-autoload DEBUG: prog: trace_ip_local_deliver_finish is made no-autoload DEBUG: prog: trace_ip_forward is made no-autoload DEBUG: prog: trace_ip_forward_finish is made no-autoload DEBUG: prog: trace_ip6_forward is made no-autoload DEBUG: prog: trace_ip6_forward is made no-autoload DEBUG: prog: __trace_ip6_rcv_finish is made no-autoload DEBUG: prog: trace_ip6_rcv_finish is made no-autoload DEBUG: prog: trace_ip6_rcv_core is made no-autoload DEBUG: prog: __trace_ip6_rcv_core is made no-autoload DEBUG: prog: trace_ipv6_rcv is made no-autoload DEBUG: prog: trace_ipv6_rcv is made no-autoload DEBUG: prog: trace_ip_output is made no-autoload DEBUG: prog: trace_ip_finish_output is made no-autoload DEBUG: prog: trace_ip_finish_output_gso is made no-autoload DEBUG: prog: trace_ip_finish_output2 is made no-autoload DEBUG: prog: trace_ip6_output is made no-autoload DEBUG: prog: trace_ip6_output is made no-autoload DEBUG: prog: trace_ip6_finish_output is made no-autoload DEBUG: prog: trace_ip6_finish_output is made no-autoload DEBUG: prog: __trace_ip6_finish_output2 is made no-autoload DEBUG: prog: trace_ip6_finish_output2 is made no-autoload DEBUG: prog: trace_ip6_send_skb is made no-autoload DEBUG: prog: __trace_ip6_send_skb is made no-autoload DEBUG: prog: trace_ip6_local_out is made no-autoload DEBUG: prog: trace_xfrm4_output is made no-autoload DEBUG: prog: trace_xfrm4_output is made no-autoload DEBUG: prog: trace_xfrm_output is made no-autoload DEBUG: prog: trace_xfrm_output is made no-autoload DEBUG: prog: trace_xfrm_output2 is made no-autoload DEBUG: prog: trace_xfrm_output2 is made no-autoload DEBUG: prog: trace_xfrm_output_gso is made no-autoload DEBUG: prog: __trace_xfrm_output_gso is made no-autoload DEBUG: prog: trace_xfrm_output_resume is made no-autoload DEBUG: prog: trace_xfrm_output_resume is made no-autoload DEBUG: prog: __trace_xfrm4_transport_output is made no-autoload DEBUG: prog: trace_xfrm4_transport_output is made no-autoload DEBUG: prog: trace_xfrm4_prepare_output is made no-autoload DEBUG: prog: __trace_xfrm4_prepare_output is made no-autoload DEBUG: prog: trace_xfrm4_policy_check is made no-autoload DEBUG: prog: trace_xfrm4_policy_check is made no-autoload DEBUG: prog: trace_xfrm4_rcv is made no-autoload DEBUG: prog: trace_xfrm4_rcv is made no-autoload DEBUG: prog: trace_xfrm_input is made no-autoload DEBUG: prog: trace_xfrm_input is made no-autoload DEBUG: prog: __trace_xfrm4_transport_input is made no-autoload DEBUG: prog: trace_xfrm4_transport_input is made no-autoload DEBUG: prog: trace_ah_output is made no-autoload DEBUG: prog: trace_ah_output is made no-autoload DEBUG: prog: trace_esp_output is made no-autoload DEBUG: prog: trace_esp_output is made no-autoload DEBUG: prog: trace_esp_output_tail is made no-autoload DEBUG: prog: __trace_esp_output_tail is made no-autoload DEBUG: prog: trace_ah_input is made no-autoload DEBUG: prog: trace_ah_input is made no-autoload DEBUG: prog: trace_esp_input is made no-autoload DEBUG: prog: trace_esp_input is made no-autoload DEBUG: prog: trace_fib_validate_source is made no-autoload DEBUG: prog: trace_ip_route_input_slow is made no-autoload DEBUG: prog: trace_ip_route_input_slow is made no-autoload DEBUG: prog: trace_tcp_v4_rcv is made no-autoload DEBUG: prog: __trace_tcp_v6_rcv is made no-autoload DEBUG: prog: trace_tcp_v6_rcv is made no-autoload DEBUG: prog: trace_tcp_filter is made no-autoload DEBUG: prog: trace_tcp_child_process is made no-autoload DEBUG: prog: trace_tcp_v4_send_reset is made no-autoload DEBUG: prog: trace_tcp_v6_send_reset is made no-autoload DEBUG: prog: trace_tcp_v6_send_reset is made no-autoload DEBUG: prog: trace_tcp_v4_do_rcv is made no-autoload DEBUG: prog: trace_tcp_v6_do_rcv is made no-autoload DEBUG: prog: trace_tcp_v6_do_rcv is made no-autoload DEBUG: prog: trace_tcp_rcv_established is made no-autoload DEBUG: prog: trace_tcp_rcv_state_process is made no-autoload DEBUG: prog: trace_tcp_queue_rcv is made no-autoload DEBUG: prog: trace_tcp_ack_probe is made no-autoload DEBUG: prog: trace_tcp_ack_probe is made no-autoload DEBUG: prog: trace_tcp_ack is made no-autoload DEBUG: prog: trace_tcp_probe_timer is made no-autoload DEBUG: prog: trace_tcp_probe_timer is made no-autoload DEBUG: prog: trace_tcp_send_probe0 is made no-autoload DEBUG: prog: __trace_tcp_send_probe0 is made no-autoload DEBUG: prog: trace_inet_lookup_listener is made no-autoload DEBUG: prog: __trace_inet6_lookup_listener is made no-autoload DEBUG: prog: trace_inet6_lookup_listener is made no-autoload DEBUG: prog: trace_tcp_transmit_skb is made no-autoload DEBUG: prog: trace_tcp_transmit_skb is made no-autoload DEBUG: prog: trace_tcp_retransmit_timer is made no-autoload DEBUG: prog: __trace_tcp_retransmit_timer is made no-autoload DEBUG: prog: trace_tcp_sendmsg_locked is made no-autoload DEBUG: prog: trace_tcp_sendmsg_locked is made no-autoload DEBUG: prog: _tracetcp_push_pending_frames is made no-autoload DEBUG: prog: trace___tcp_push_pending_frames is made no-autoload DEBUG: prog: trace_inet_listen is made no-autoload DEBUG: prog: trace_inet_listen is made no-autoload DEBUG: prog: trace_tcp_write_timer_handler is made no-autoload DEBUG: prog: trace_tcp_write_timer_handler is made no-autoload DEBUG: prog: trace_tcp_v4_destroy_sock is made no-autoload DEBUG: prog: trace_tcp_v4_destroy_sock is made no-autoload DEBUG: prog: trace_tcp_close is made no-autoload DEBUG: prog: trace_tcp_close is made no-autoload DEBUG: prog: trace_udp_rcv is made no-autoload DEBUG: prog: trace_udp_unicast_rcv_skb is made no-autoload DEBUG: prog: trace_udp_queue_rcv_skb is made no-autoload DEBUG: prog: trace_xfrm4_udp_encap_rcv is made no-autoload DEBUG: prog: trace_xfrm4_udp_encap_rcv is made no-autoload DEBUG: prog: trace_xfrm4_rcv_encap is made no-autoload DEBUG: prog: __trace_xfrm4_rcv_encap is made no-autoload DEBUG: prog: trace_udp_queue_rcv_skb is made no-autoload DEBUG: prog: trace_udp_queue_rcv_skb is made no-autoload DEBUG: prog: trace_udp_enqueue_schedule_skb is made no-autoload DEBUG: prog: trace_icmp_rcv is made no-autoload DEBUG: prog: trace_icmp_echo is made no-autoload DEBUG: prog: trace_icmp_reply is made no-autoload DEBUG: prog: trace_icmpv6_rcv is made no-autoload DEBUG: prog: trace_icmpv6_rcv is made no-autoload DEBUG: prog: trace_icmpv6_echo_reply is made no-autoload DEBUG: prog: __trace_icmpv6_echo_reply is made no-autoload DEBUG: prog: trace_ping_rcv is made no-autoload DEBUG: prog: trace___ping_queue_rcv_skb is made no-autoload DEBUG: prog: trace_ping_queue_rcv_skb is made no-autoload DEBUG: prog: trace_ping_queue_rcv_skb is made no-autoload DEBUG: prog: trace_ping_lookup is made no-autoload DEBUG: prog: trace_kfree_skb is made no-autoload DEBUG: eBPF is loaded successfully DEBUG: begin to attach eBPF program... libbpf: prog 'trace_netif_receive_skb_core': can't attach BPF program w/o FD (did you load it?) libbpf: prog 'trace_netif_receive_skb_core': failed to attach to kprobe 'netif_receive_skb_core.constprop.0+0x0': Invalid argument WARN: retring to attach in legacy mode, prog=trace___netif_receive_skb_core, func=netif_receive_skb_core.constprop.0 manually attach prog trace___netif_receive_skb_core success manually attach prog trace_netif_receive_skb_core_pskb success manually attach prog __trace_ip_rcv_core success manually attach prog trace_icmp_reply success manually attach prog __trace_ping_lookup success DEBUG: eBPF program attached successfully begin trace...
一直卡在这里,我是从另外一个终端向它ping包的,网络可以ping通
还要确保CONFIG_KALLSYMS处于打开状态。nettrace需要查询符号是否存在,来决定是否进行函数的跟踪,需要依赖于这个内核配置哦
我刚查了一下内核配置,CONFIG_KALLSYMS是打开的
上述打印中最后 DEBUG: begin to attach eBPF program... libbpf: prog 'trace___netif_receive_skb_core': can't attach BPF program w/o FD (did you load it?) libbpf: prog 'trace_netif_receive_skb_core': failed to attach to kprobe 'netif_receive_skb_core.constprop.0+0x0': Invalid argument WARN: retring to attach in legacy mode, prog=trace___netif_receive_skb_core, func=netif_receive_skb_core.constprop.0 manually attach prog trace___netif_receive_skb_core success manually attach prog trace_netif_receive_skb_core_pskb success manually attach prog __trace_ip_rcv_core success manually attach prog trace_icmp_reply success manually attach prog __trace_ping_lookup success DEBUG: eBPF program attached successfully begin trace...
这个有failed, 有影响不?
这个失败是没有问题的。可以用bpftool prog show查看一下是否BPF程序都加载成功了。
另外,请问用的是COMPAT模式编译的吗?是的话,要确保编译使用的头文件和当前所 运行的内核是一致的。
bpftool prog show 在嵌入式环境还没有。 是COMPAT模式编译的,不过编译主机跟运行主机可能内核头文件不一致。 是不是make KERNEL=/home/ubuntu/kernel COMPAT=1 all 这样编译?
最新代码,用容器编译,还是有如下错误(aarch64)
gcc trace.c /root/nettrace/shared/pkt_utils.c /root/nettrace/component/net_utils.c /root/nettrace/component/arg_parse.c /root/nettrace/component/sys_utils.c /root/nettrace/shared/bpf_utils.c trace_probe.c trace_tracing.c analysis.c /root/nettrace/component/parse_sym.c trace_group.c dropreason.c nettrace.c -o nettrace -lbpf -lelf -lz -O2 -static -I./ -I/root/nettrace/shared/bpf/ -DCOMPAT_MODE -DBPF_FEAT_STACK_TRACE -Wno-deprecated-declarations -DVERSION=1.2.5 -DRELEASE=.tl3 -I/root/nettrace/shared/ -I/root/nettrace/component
trace_probe.c: In function 'probe_trace_load':
trace_probe.c:134:38: error: 'struct kprobe' has no member named 'maps'
134 | bpf_mapset_max_entries(skel->maps.m_event, get_nprocs_conf());
| ^~
In file included from trace.h:12,
from trace_probe.c:4:
/root/nettrace/shared/bpf_utils.h:25:34: error: 'struct kprobe' has no member named 'maps'
25 | int fd = bpf_mapfd(skel->maps.m_config); \
| ^~
trace_probe.c:142:9: note: in expansion of macro 'bpf_set_config'
142 | bpf_set_config(skel, bss, trace_ctx.bpf_args);
| ^~~~~~
trace_probe.c: In function 'probe_trace_ready':
/root/nettrace/shared/bpf_utils.h:40:34: error: 'struct kprobe' has no member named 'maps'
40 | int fd = bpf_mapfd(skel->maps.m_config); \
| ^~
trace_probe.c:271:9: note: in expansion of macro 'bpf_set_config_field'
271 | bpf_set_config_field(skel, bss, ready, true);
| ^~~~~~~~
trace_probe.c: In function 'probe_print_stack':
trace_probe.c:277:38: error: 'struct kprobe' has no member named 'maps'
277 | int map_fd = bpf_mapfd(skel->maps.m_stack);
| ^~
make[1]: *** [../common.mk:103: nettrace] Error 1
make[1]: Leaving directory '/root/nettrace/src'
make[1]: Entering directory '/root/nettrace/legacy'
x86_64镜像没此编译问题,是否arm64的容器镜像不一致?
sorry,测试了一下,最新的代码arm64上编译的确有点问题。。。。。。
待我修复一下~
内核版本在5.4以上,内核配置又开启了CONFIG_DEBUG_INFO_BTF=y的话,是不需要加COMPAT的,这样编译出来的版本就是通用版本。
问题已修复,在ubuntu20.04 arm上已经进行过了测试,编译没问题~
为啥我这边还有?确认aarm64镜像你那边要不要上传一个?
gcc trace.c /root/nettrace/shared/pkt_utils.c /root/nettrace/component/net_utils.c /root/nettrace/component/arg_parse.c /root/nettrace/component/sys_utils.c /root/nettrace/shared/bpf_utils.c trace_probe.c trace_tracing.c analysis.c /root/nettrace/component/parse_sym.c trace_group.c dropreason.c nettrace.c -o nettrace -lbpf -lelf -lz -O2 -static -I./ -I/root/nettrace/shared/bpf/ -DCOMPAT_MODE -DBPF_FEAT_STACK_TRACE -Wno-deprecated-declarations -DVERSION=1.2.5 -DRELEASE=.tl3 -I/root/nettrace/shared/ -I/root/nettrace/component
trace_probe.c: In function 'probe_trace_load':
trace_probe.c:134:38: error: 'struct kprobe' has no member named 'maps'
134 | bpf_mapset_max_entries(skel->maps.m_event, get_nprocs_conf());
| ^~
In file included from trace.h:12,
from trace_probe.c:4:
/root/nettrace/shared/bpf_utils.h:25:34: error: 'struct kprobe' has no member named 'maps'
25 | int fd = bpf_mapfd(skel->maps.m_config); \
| ^~
trace_probe.c:142:9: note: in expansion of macro 'bpf_set_config'
142 | bpf_set_config(skel, bss, trace_ctx.bpf_args);
| ^~~~~~
trace_probe.c: In function 'probe_trace_ready':
/root/nettrace/shared/bpf_utils.h:40:34: error: 'struct kprobe' has no member named 'maps'
40 | int fd = bpf_mapfd(skel->maps.m_config); \
| ^~
trace_probe.c:271:9: note: in expansion of macro 'bpf_set_config_field'
271 | bpf_set_config_field(skel, bss, ready, true);
| ^~~~~~~~
trace_probe.c: In function 'probe_print_stack':
trace_probe.c:277:38: error: 'struct kprobe' has no member named 'maps'
277 | int map_fd = bpf_mapfd(skel->maps.m_stack);
| ^~
重新编译之前请clean一下
我是这样试的
其中/root/kernel 是我本地arm64的kernel 头文件
看错误应该是最新的代码trace_probe.c 结构体kprobe没有maps成员 trace_probe.c:134:38: error: 'struct kprobe' has no member named 'maps'
这个报错大概率是之前的BPF程序编译已经出错了,不是第一现场。需要看一下 BPF程序编译失败的原因,大概率和你的头文件有关系。
你的编译方式看起来是没问题的。
实在不行,不要用COMPAT模式的,可以用BTF版本的嘛,你的内核版本是足够高的。
主要是要交叉编译,运行的设备的内核头文件跟编译系统的头文件不一致。
我去掉COMPAT=1 并且代码退回到v1.2.5 可以编译出来,但运行ERROR: BTF is not support by your kernel, please compilethis tool with "COMPAT=1"
验证可行,直接用编译好的,因为我的内核版本够高。谢谢
内核版本在5.4以上,内核配置又开启了CONFIG_DEBUG_INFO_BTF=y的话,是不需要加COMPAT的,这样编译出来的版本就是通用版本。
问题已修复,在ubuntu20.04 arm上已经进行过了测试,编译没问题~
内核版本这里,ICMPV6_EXT_ECHO_REQUEST
和ICMPV6_EXT_ECHO_REPLY
这两个宏似乎出现在5.13以后,可以做个小修正。
https://github.com/torvalds/linux/commit/750f4fc2a12f6632b5aa04526bf57fa06bfe8467
ICMPV6_EXT_ECHO_REPLY
哈喽,这里是编译有遇到什么问题吗?这两个宏定义在skb_shared.h里面已经做了适配了的:
/* Codes for EXT_ECHO (PROBE) */
#ifndef ICMPV6_EXT_ECHO_REQUEST
#define ICMPV6_EXT_ECHO_REQUEST 160
#endif
#ifndef ICMPV6_EXT_ECHO_REPLY
#define ICMPV6_EXT_ECHO_REPLY 161
#endif
ICMPV6_EXT_ECHO_REPLY
哈喽,这里是编译有遇到什么问题吗?这两个宏定义在skb_shared.h里面已经做了适配了的:
/* Codes for EXT_ECHO (PROBE) */ #ifndef ICMPV6_EXT_ECHO_REQUEST #define ICMPV6_EXT_ECHO_REQUEST 160 #endif #ifndef ICMPV6_EXT_ECHO_REPLY #define ICMPV6_EXT_ECHO_REPLY 161 #endif
没有哈,是之前用老版本内核编译不过然后查了下资料,确认了下最低版本是多少。
如果非要说编译不过的话,我之前用腾讯云上这个OpenCloudOS是编译不过的,这个内核版本应该也是5.4。
Hi : 我的系统是ubuntu 20.04LTS,安装了sudo apt install libelf-dev libbpf-dev linux-headers-
uname -r
clang llvm gcc linux-tools-uname -r
linux-tools-generic -ymake命令编译的时候出现 rm progs/ntrace.o make[1]: Leaving directory '/home/zhiye/tools/sourcecodebuild/nettrace/nodetrace' make[1]: Entering directory '/home/zhiye/tools/sourcecodebuild/nettrace/src' python3 gen_trace.py > trace_group.c ln -s ../shared/bpf/vmlinux.h kheaders.h clang -O2 -c -g -S -Wall -Wno-pointer-sign -Wno-unused-value \ -Wno-incompatible-pointer-types-discards-qualifiers \ -fno-asynchronous-unwind-tables \ progs/tracing.c -emit-llvm -Wno-unknown-attributes -I./ -I/home/zhiye/tools/sourcecodebuild/nettrace/shared/bpf/ -DSTACK_TRACE -Wno-unused-function -Xclang \ -disable-llvm-passes -o - | \ opt -O2 -mtriple=bpf-pc-linux | \ llvm-dis | \ llc -march=bpf -filetype=obj -o progs/tracing.o bpftool gen skeleton progs/tracing.o > progs/tracing.skel.h /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) python3 gen_trace.py probe > progs/kprobe_trace.h clang -O2 -c -g -S -Wall -Wno-pointer-sign -Wno-unused-value \ -Wno-incompatible-pointer-types-discards-qualifiers \ -fno-asynchronous-unwind-tables \ progs/kprobe.c -emit-llvm -Wno-unknown-attributes -I./ -I/home/zhiye/tools/sourcecodebuild/nettrace/shared/bpf/ -DSTACK_TRACE -Wno-unused-function -Xclang \ -disable-llvm-passes -o - | \ opt -O2 -mtriple=bpf-pc-linux | \ llvm-dis | \ llc -march=bpf -filetype=obj -o progs/kprobe.o bpftool gen skeleton progs/kprobe.o > progs/kprobe.skel.h /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) gcc trace.c /home/zhiye/tools/sourcecodebuild/nettrace/shared/pkt_utils.c /home/zhiye/tools/sourcecodebuild/nettrace/component/net_utils.c /home/zhiye/tools/sourcecodebuild/nettrace/component/arg_parse.c /home/zhiye/tools/sourcecodebuild/nettrace/component/sys_utils.c /home/zhiye/tools/sourcecodebuild/nettrace/shared/bpf_utils.c trace_probe.c trace_tracing.c analysis.c /home/zhiye/tools/sourcecodebuild/nettrace/component/parse_sym.c trace_group.c dropreason.c nettrace.c -o nettrace -lbpf -lelf -lz -g -O2 -static -I./ -I/home/zhiye/tools/sourcecodebuild/nettrace/shared/bpf/ -DSTACK_TRACE -Wno-deprecated-declarations -DVERSION=1.2.5 -DRELEASE=.tl3 -I/home/zhiye/tools/sourcecodebuild/nettrace/shared/ -I/home/zhiye/tools/sourcecodebuild/nettrace/component /usr/bin/ld: /tmp/ccs5ijfW.o: in function
probe_trace_attach_manual': /home/zhiye/tools/sourcecodebuild/nettrace/src/trace_probe.c:34: undefined reference to
bpf_program__set_autoattach' collect2: error: ld returned 1 exit status make[1]: *** [../common.mk:100: nettrace] Error 1