nf_conntrack: table full, dropping packet.

[dyrnq]

kube-proxy.log

I1228 08:37:04.500646       1 node.go:172] Successfully retrieved node IP: 192.168.26.21
I1228 08:37:04.500790       1 server_others.go:140] Detected node IP 192.168.26.21
I1228 08:37:04.548738       1 server_others.go:206] kube-proxy running in dual-stack mode, IPv4-primary
I1228 08:37:04.548841       1 server_others.go:274] Using ipvs Proxier.
I1228 08:37:04.548888       1 server_others.go:276] creating dualStackProxier for ipvs.
W1228 08:37:04.548922       1 server_others.go:495] detect-local-mode set to ClusterCIDR, but no IPv6 cluster CIDR defined, , defaulting to no-op detect-local for IPv6
W1228 08:37:04.549451       1 proxier.go:445] IPVS scheduler not specified, use rr by default
W1228 08:37:04.549844       1 proxier.go:445] IPVS scheduler not specified, use rr by default
W1228 08:37:04.549954       1 ipset.go:113] ipset name truncated; [KUBE-6-LOAD-BALANCER-SOURCE-CIDR] -> [KUBE-6-LOAD-BALANCER-SOURCE-CID]
W1228 08:37:04.549986       1 ipset.go:113] ipset name truncated; [KUBE-6-NODE-PORT-LOCAL-SCTP-HASH] -> [KUBE-6-NODE-PORT-LOCAL-SCTP-HAS]
I1228 08:37:04.550242       1 server.go:647] Version: v1.21.8
I1228 08:37:04.553318       1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_max' to 131072
I1228 08:37:04.553459       1 conntrack.go:52] Setting nf_conntrack_max to 131072
I1228 08:37:04.553849       1 conntrack.go:83] Setting conntrack hashsize to 32768
I1228 08:37:04.570740       1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
I1228 08:37:04.570973       1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
I1228 08:37:04.572175       1 config.go:315] Starting service config controller
I1228 08:37:04.572254       1 shared_informer.go:240] Waiting for caches to sync for service config
I1228 08:37:04.572450       1 config.go:224] Starting endpoint slice config controller
I1228 08:37:04.572490       1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
W1228 08:37:04.577068       1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
W1228 08:37:04.582666       1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
I1228 08:37:04.673735       1 shared_informer.go:247] Caches are synced for endpoint slice config 
I1228 08:37:04.676999       1 shared_informer.go:247] Caches are synced for service config 

(Optional) By default, kube-proxy sets the nf_conntrack_max kernel parameter to a default value that may differ from what os originally sets at boot. To keep os`s default setting, edit the kube-proxy configuration with the following command.

kubectl edit -n kube-system daemonset kube-proxy

Add --conntrack-max-per-core and --conntrack-min to the kube-proxy arguments as shown in the following example. A setting of 0 implies no change.

containers:
      - command:
        - kube-proxy
        - --v=2
        - --config=/var/lib/kube-proxy-config/config
        - --conntrack-max-per-core=0
        - --conntrack-min=0

kube-proxy

--conntrack-max-per-core int32     Default: 32768
  | Maximum number of NAT connections to track per CPU core (0 to leave the limit as-is and ignore conntrack-min).
--conntrack-min int32     Default: 131072
  | Minimum number of conntrack entries to allocate, regardless of conntrack-max-per-core (set conntrack-max-per-core=0 to leave the limit as-is).

[dyrnq]

It looks like ip_conntrack hashsize = --conntrack-max-per-core and net.netfilter.nf_conntrack_max = --conntrack-max-per-core * cpu

add to the /etc/modprobe.conf
options ip_conntrack hashsize=32768

[dyrnq]

ip_conntrack hashsize = max /4 https://github.com/kubernetes/kubernetes/blob/v1.23.1/cmd/kube-proxy/app/server.go#L794 https://github.com/kubernetes/kubernetes/blob/v1.23.1/cmd/kube-proxy/app/conntrack.go#L49