search

sonic-net / sonic-host-services

4 stars 80 forks source link

Disable connection tracking for icmpv6 traffic #160

Closed prabhataravind closed 2 months ago

prabhataravind commented 2 months ago

icmpv6 connection tracking can cause conntrack table in kernel to grow rapidly and lead to packets being dropped, making the device unresponsive to connection requests. The signature seen when this happens is the following:

"kernel: nf_conntrack: nf_conntrack: table full, dropping packet”

An investigation on this identified that most of the entries in conntrack table were due to unreplied icmpv6 requests which ended up in the dying list.

Snippet of "conntrack -f ipv6 -L dying " on a device with the issue:

icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:3de0:f090:e3e0 type=128 code=0 id=56516 [UNREPLIED] src=2603:10b0:105:1065:0:3de0:f090:e3e0 dst=2603:10b0:105:1065::1 type=129 code=0 id=56516 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:8:9e9a:fa90:e3d0 type=128 code=0 id=49585 [UNREPLIED] src=2603:10b0:105:1065:8:9e9a:fa90:e3d0 dst=2603:10b0:105:1065::1 type=129 code=0 id=49585 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:1:6af9:fd90:e3cc type=128 code=0 id=50650 [UNREPLIED] src=2603:10b0:105:1065:1:6af9:fd90:e3cc dst=2603:10b0:105:1065::1 type=129 code=0 id=50650 mark=0 use=2 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:1:21fb:f090:e3c4 type=128 code=0 id=37425 [UNREPLIED] src=2603:10b0:105:1065:1:21fb:f090:e3c4 dst=2603:10b0:105:1065::1 type=129 code=0 id=37425 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:8715:f690:e3cc type=128 code=0 id=49153 [UNREPLIED] src=2603:10b0:105:1065:0:8715:f690:e3cc dst=2603:10b0:105:1065::1 type=129 code=0 id=49153 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:4b95:f790:e3c4 type=128 code=0 id=12475 [UNREPLIED] src=2603:10b0:105:1065:0:4b95:f790:e3c4 dst=2603:10b0:105:1065::1 type=129 code=0 id=12475 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:453e:f090:e3d4 type=128 code=0 id=57081 [UNREPLIED] src=2603:10b0:105:1065:0:453e:f090:e3d4 dst=2603:10b0:105:1065::1 type=129 code=0 id=57081 mark=0 use=1

The fix here is to use the raw table PREROUTING and OUTPUT chains in ip6tables to disable CT for icmpv6 packets as these really don't need to be tracked. SONiC has other iptables rules below to accept icmpv6 traffic even without explicit tracking

-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 128 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 129 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 1 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 3 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 136 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 133 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 134 -j ACCEPT

The change has been tested manually using ipv6 neighbors in INCOMPLETE/FAILED states and verifying that the icmpv6 connection requests do not get added to the kernel conntrack table. The change has also been validated against arp/ndp/ cacl/nat sonic-mgmt test cases as well. A sonic-mgmt test gap is also opened to automate tests for this scenario.

StormLiangMS commented 2 months ago

@yxieca for cherry pick.