Can't ping another pod across the nodes via IPIPCrossSubnet

[631068264]

apply this deploy

apiVersion: apps/v1

kind: Deployment

metadata:

  name: dnsutils
  labels:
    app: dnsutils
spec:
  selector:
    matchLabels:
      app: dnsutils
  replicas: 3

  template:
    metadata:
      labels:
        app: dnsutils
    spec:

      containers:

      - name: dnsutils
        image: mydlqclub/dnsutils:1.3
        command: ["sleep","3600"]

Node 155 57 53 in the same subnet

dnsutils-67865b6fb5-j5mg8   1/1     Running   0          15s   10.42.221.130   xxx.155   <none>           <none>
dnsutils-67865b6fb5-xxs79   1/1     Running   0          15s   10.42.139.134   xxx.57    <none>           <none>
dnsutils-67865b6fb5-z9vvd   1/1     Running   0          15s   10.42.225.196   xxx.53    <none>           <none>

Only pod in Node A can ping itself , can't ping pod in NodeB or NodeC and other dnsutils pod same

 kubectl exec -it pod/dnsutils-67865b6fb5-j5mg8 -- /bin/sh

traceroute 10.42.139.134
traceroute to 10.42.139.134 (10.42.139.134), 30 hops max, 46 byte packets
 1  xxx-155.kubernetes.default.svc.cluster.local (xxx.155)  0.008 ms  0.004 ms  0.004 ms
 2  *  *

  *
 3  *  *  *
 4  *

on xxx.155

ip route|grep bird

10.42.139.128/26 via xxxx.57 dev eth0 proto bird 
blackhole 10.42.221.128/26 proto bird 
10.42.225.192/26 via xxx.53 dev eth0 proto bird 

kubectl get installation -o yaml

apiVersion: v1
items:
- apiVersion: operator.tigera.io/v1
  kind: Installation
.......
  spec:
    calicoNetwork:
      bgp: Enabled
      hostPorts: Enabled
      ipPools:
      - blockSize: 26
        cidr: 10.42.0.0/16
        disableBGPExport: false
        encapsulation: IPIPCrossSubnet
        natOutgoing: Enabled
        nodeSelector: all()
      linuxDataplane: Iptables
      multiInterfaceMode: None
      nodeAddressAutodetectionV4:
        firstFound: true
    cni:
      ipam:
        type: Calico
      type: Calico
    controlPlaneReplicas: 2
    flexVolumePath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/
    imagePullSecrets: []
    kubeletVolumePluginPath: /var/lib/kubelet
    kubernetesProvider: ""
    nodeUpdateStrategy:
      rollingUpdate:
        maxUnavailable: 1
      type: RollingUpdate
    nonPrivileged: Disabled
    variant: Calico

If Use encapsulation: IPIP It can ping

Context

Your Environment

• Calico version 3.25 operator
• Orchestrator version (e.g. kubernetes, mesos, rkt):kubernetes 1.23.6
• Operating System and version: Centos7
• Link to your project (optional):

calicoctl node diags

diags-20230313_215827.tar.gz

[sridhartigera]

@631068264 Do you see the packets at the destination node?

[631068264]

Accroding to this , no packets at the destination node. This pod is on 155 node. Follow the route it can't reach 57 the destination node

 kubectl exec -it pod/dnsutils-67865b6fb5-j5mg8 -- /bin/sh

traceroute 10.42.139.134
traceroute to 10.42.139.134 (10.42.139.134), 30 hops max, 46 byte packets
 1  xxx-155.kubernetes.default.svc.cluster.local (xxx.155)  0.008 ms  0.004 ms  0.004 ms
 2  *  *

  *
 3  *  *  *
 4  *

[sridhartigera]

@631068264 Do the nodes have src-dest check turned on? For example, in aws src-dest check needs to be turned off for this to work.

[sridhartigera]

Closing this due to inactivity. Feel free to reopen if there is any update.