K8s Calico natOutGoing is not work on v3.28.2

[sunminming]

When I curl an External Services in pod, the curl is timeout, with tcpdump, snat is not work, the source ip is endpoint ip in pod

root@k8s-10-10-40-34:/etc/cni/net.d# calicoctl get ippool -o yaml
apiVersion: projectcalico.org/v3
items:
- apiVersion: projectcalico.org/v3
  kind: IPPool
  metadata:
    creationTimestamp: "2024-11-17T15:37:45Z"
    name: default-ipv4-ippool
    resourceVersion: "6080"
    uid: a5ab5ddd-d6f3-4edf-b99a-5d2097ba16a5
  spec:
    allowedUses:
    - Workload
    - Tunnel
    blockSize: 26
    cidr: 172.20.0.0/16
    ipipMode: Always
    natOutgoing: true
    nodeSelector: all()
    vxlanMode: Never
kind: IPPoolList
metadata:
  resourceVersion: "380709"

root@k8s-10-10-40-34:/etc# tcpdump -i enp6s18 host 10.10.40.37 -vvvnnn
tcpdump: listening on enp6s18, link-type EN10MB (Ethernet), snapshot length 262144 bytes
23:33:31.165813 IP (tos 0x0, ttl 63, id 14947, offset 0, flags [DF], proto TCP (6), length 60)
    172.20.71.28.38918 > 10.10.40.37.80: Flags [S], cksum 0x258e (incorrect -> 0x8240), seq 2520064448, win 64240, options [mss 1460,sackOK,TS val 2294271075 ecr 0,nop,wscale 7], length 0
23:33:32.186596 IP (tos 0x0, ttl 63, id 14948, offset 0, flags [DF], proto TCP (6), length 60)
    172.20.71.28.38918 > 10.10.40.37.80: Flags [S], cksum 0x258e (incorrect -> 0x7e43), seq 2520064448, win 64240, options [mss 1460,sackOK,TS val 2294272096 ecr 0,nop,wscale 7], length 0
23:33:33.210686 IP (tos 0x0, ttl 63, id 14949, offset 0, flags [DF], proto TCP (6), length 60)
    172.20.71.28.38918 > 10.10.40.37.80: Flags [S], cksum 0x258e (incorrect -> 0x7a43), seq 2520064448, win 64240, options [mss 1460,sackOK,TS val 2294273120 ecr 0,nop,wscale 7], length 0
23:33:34.238594 IP (tos 0x0, ttl 63, id 14950, offset 0, flags [DF], proto TCP (6), length 60)
    172.20.71.28.38918 > 10.10.40.37.80: Flags [S], cksum 0x258e (incorrect -> 0x763f), seq 2520064448, win 64240, options [mss 1460,sackOK,TS val 2294274148 ecr 0,nop,wscale 7], length 0

Expected Behavior

source ip is node ip

Current Behavior

source ip is pod ip

Possible Solution

Steps to Reproduce (for bugs)

curl an External Server in pod

Context

Your Environment

• Calico version v3.28,2
• Calico dataplane (iptables, windows etc.) iptables, and kube-proxy is iptables mode
• Orchestrator version (e.g. kubernetes, mesos, rkt):
• Operating System and version: ubuntu 22.04
• Link to your project (optional):