OpenCloudOS / nettrace

nettrace is a eBPF-based tool to trace network packet and diagnose network problem.
Other
326 stars 80 forks source link

add option to show kfree_skb call stack #8

Closed curu closed 1 year ago

curu commented 2 years ago

For kernel versions that do not support drop reason, or drop point that has not yet been traced, it may be helpful to show the call stack of kfree_skb, so that we can infer the drop reason.

eg:

nettrace -p udp --diag  --diag-quiet --diag-stack
...
[1] ERROR happens in kfree_skb(life):
    packet is dropped by kernel
    location:
    __udp4_lib_rcv+0x8fe
    stack:
       __udp4_lib_rcv ([kernel.kallsyms])
       ip_protocol_deliver_rcu ([kernel.kallsyms])
       ip_local_deliver_finish ([kernel.kallsyms])
       ip_local_deliver ([kernel.kallsyms])
       ip_rcv_finish ([kernel.kallsyms])
       ip_rcv ([kernel.kallsyms])
       __netif_receive_skb_core ([kernel.kallsyms])
       __netif_receive_skb_list_core ([kernel.kallsyms])
       netif_receive_skb_list_internal ([kernel.kallsyms])
       gro_normal_list.part.143 ([kernel.kallsyms])
       napi_complete_done ([kernel.kallsyms])
       virtnet_poll ([kernel.kallsyms])
       net_rx_action ([kernel.kallsyms])
menglongdong commented 1 year ago

已完成支持:

nettrace -p tcp --drop --drop-stack 
begin trace...
[2114.791575] TCP: 127.0.0.1:46544 -> 127.0.0.1:9999 seq:714874229, ack:0, flags:S, tcp_v4_rcv+0x63
Call Stack:
    -> kfree_skb_reason+0xb8
    -> kfree_skb_reason+0xb8
    -> tcp_v4_rcv+0x63
    -> ip_protocol_deliver_rcu+0x30
    -> ip_local_deliver_finish+0x48
    -> ip_local_deliver+0xcb
    -> ip_rcv_finish+0x83
    -> ip_rcv+0xb1
    -> __netif_receive_skb_one_core+0x85
    -> process_backlog+0x9e
    -> napi_poll+0x9d
    -> net_rx_action+0xac
    -> __softirqentry_text_start+0xdd
    -> do_softirq_own_stack+0x2a
    -> do_softirq+0x41
    -> __local_bh_enable_ip+0x50
    -> ip_finish_output2+0x1be
    -> __ip_finish_output+0xb0
    -> ip_finish_output+0x36
    -> ip_output+0x6b
    -> __ip_queue_xmit+0x18c
    -> ip_queue_xmit+0x10
    -> __tcp_transmit_skb+0x528
    -> tcp_connect+0x34e
    -> tcp_v4_connect+0x3d8
    -> __inet_stream_connect+0xd3
    -> inet_stream_connect+0x3b
    -> __sys_connect+0xab
    -> __x64_sys_connect+0x18
    -> do_syscall_64+0x49
    -> entry_SYSCALL_64_after_hwframe+0x44