search

cilium / pwru

Packet, where are you? -- eBPF-based Linux kernel networking debugger
Apache License 2.0
2.8k stars 171 forks source link

pwru doesn't capture ip_forwarding failure #416

Open barani129 opened 2 months ago

barani129 commented 2 months ago

We had an issue with one of the clusters where IP forwarding is disabled by default in the k8s worker nodes. And ran pwru with detailed filtered options, but couldn't see skb_drop reason due to ip forwarding. Is it expected?

Kernel version: 5.14.0-284.66.1.el9_2.x86_64

10.14.134.10 is the external IP and .20 is worker IP on storage subnet. It would have forwarded the packet to default ovn network in k8s if ip forwarding had been enabled, but in the trace below, after skb_push couldn't see any reason for failure.

./pwru 'host 10.14.134.10' --output-skb --output-meta --output-tuple

0xff452c7c65ea9e00 4 :0 4026531840 0 ens1f0:2 0x0800 9000 74 10.14.134.20:35676->10.14.134.10:443(tcp) napi_consume_skb ice_clean_tx_irq[ice] 0xff452c7c65ea9e00 4 :0 4026531840 0 ens1f0:2 0x0800 9000 74 10.14.134.20:35676->10.14.134.10:443(tcp) skb_release_head_state napi_consume_skb 0xff452c7c65ea9e00 4 :0 4026531840 0 ens1f0:2 0x0800 9000 74 10.14.134.20:35676->10.14.134.10:443(tcp) skb_release_data napi_consume_skb 0xff452c7c65ea9e00 4 :0 4026531840 0 ens1f0:2 0x0800 9000 74 10.14.134.20:35676->10.14.134.10:443(tcp) skb_free_head skb_release_data 0xff452c7de75fc000 10 :0 4026531840 0 bond0.38:17 0x0800 9000 60 10.14.134.10:443->10.14.134.20:35676(tcp) skb_push vlan_do_receive 0xff452c7de75fc000 10 :0 4026531840 0 bond0.38:17 0x0800 9000 74 10.14.134.10:443->10.14.134.20:35676(tcp) skb_push vlan_do_receive 0xff452c7de75fe000 10 :0 4026531840 0 bond0.38:17 0x0800 9000 60 10.14.134.10:443->10.14.134.20:35676(tcp) skb_push vlan_do_receive 0xff452c7de75fe000 10 :0 4026531840 0 bond0.38:17 0x0800 9000 74 10.14.134.10:443->10.14.134.20:35676(tcp) skb_push vlan_do_receive

brb commented 1 month ago

The 5.14.0 kernel might not have the SKB drop reasons when IP forwarding is disabled.