search

ovn-org / ovn-kubernetes

A robust Kubernetes networking platform
https://ovn-kubernetes.io/
Apache License 2.0
824 stars 347 forks source link

`Should validate connectivity within a namespace of pods on separate nodes` flakes on V6 #4185

Open tssurya opened 7 months ago

tssurya commented 7 months ago

sample failure: https://github.com/ovn-org/ovn-kubernetes/actions/runs/8053130165/job/21995745660?pr=4106 https://github.com/ovn-org/ovn-kubernetes/pull/4106#issuecomment-1965916181 

2024-02-26T19:41:36.5352295Z test e2e inter-node connectivity between worker nodes Should validate connectivity within a namespace of pods on separate nodes
2024-02-26T19:41:36.5354298Z /home/runner/work/ovn-kubernetes/ovn-kubernetes/test/e2e/e2e.go:980
2024-02-26T19:41:36.5355913Z   STEP: Creating a kubernetes client @ 02/26/24 19:41:36.532
2024-02-26T19:41:36.5356992Z   Feb 26 19:41:36.532: INFO: >>> kubeConfig: /home/runner/ovn.conf
2024-02-26T19:41:36.5358027Z   STEP: Building a namespace api object, basename inter-node-e2e @ 02/26/24 19:41:36.533
2024-02-26T19:41:36.5409733Z   Feb 26 19:41:36.540: INFO: Skipping waiting for service account
2024-02-26T19:41:36.5411775Z   STEP: Creating a container on node ovn-worker and verifying connectivity to a pod on node ovn-worker2 @ 02/26/24 19:41:36.54
2024-02-26T19:41:36.5511985Z   W0226 19:41:36.549766   62222 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "e2e-dst-ping-pod-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "e2e-dst-ping-pod-container" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "e2e-dst-ping-pod-container" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "e2e-dst-ping-pod-container" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
2024-02-26T19:41:40.5613825Z   Feb 26 19:41:40.561: INFO: Running '/usr/local/bin/kubectl --server=https://[::1]:38175 --kubeconfig=/home/runner/ovn.conf --namespace=inter-node-e2e-196 get pods e2e-dst-ping-pod --template={{.status.podIP}}'
2024-02-26T19:41:40.6114316Z   Feb 26 19:41:40.611: INFO: stderr: ""
2024-02-26T19:41:40.6120275Z   Feb 26 19:41:40.611: INFO: stdout: "fd00:10:244:3::36"
2024-02-26T19:41:40.6121867Z   Feb 26 19:41:40.611: INFO: Destination ping target for e2e-dst-ping-pod is fd00:10:244:3::36
2024-02-26T19:41:40.6194914Z   W0226 19:41:40.615979   62222 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "e2e-src-ping-pod-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "e2e-src-ping-pod-container" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "e2e-src-ping-pod-container" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "e2e-src-ping-pod-container" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
2024-02-26T19:42:20.6263942Z   [FAILED] in [It] - /home/runner/work/ovn-kubernetes/ovn-kubernetes/test/e2e/e2e.go:189 @ 02/26/24 19:42:20.626
2024-02-26T19:42:21.2987336Z   STEP: dump namespace information after failure @ 02/26/24 19:42:21.298
2024-02-26T19:42:21.2988751Z   STEP: Collecting events from namespace "inter-node-e2e-196". @ 02/26/24 19:42:21.298
2024-02-26T19:42:21.3012489Z   STEP: Found 5 events. @ 02/26/24 19:42:21.301
2024-02-26T19:42:21.3015052Z   Feb 26 19:42:21.301: INFO: At 2024-02-26 19:41:38 +0000 UTC - event for e2e-dst-ping-pod: {kubelet ovn-worker2} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.26" already present on machine
2024-02-26T19:42:21.3018162Z   Feb 26 19:42:21.301: INFO: At 2024-02-26 19:41:38 +0000 UTC - event for e2e-dst-ping-pod: {kubelet ovn-worker2} Created: Created container e2e-dst-ping-pod-container
2024-02-26T19:42:21.3026272Z   Feb 26 19:42:21.301: INFO: At 2024-02-26 19:41:38 +0000 UTC - event for e2e-dst-ping-pod: {kubelet ovn-worker2} Started: Started container e2e-dst-ping-pod-container
2024-02-26T19:42:21.3032924Z   Feb 26 19:42:21.301: INFO: At 2024-02-26 19:41:40 +0000 UTC - event for e2e-src-ping-pod: {kubelet ovn-worker} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1f7f6a49a63617dc5788e7c9bce2d8a79ee54d0cd1090cd9e72652e38128b5cd": plugin type="ovn-k8s-cni-overlay" name="ovn-kubernetes" failed (add): failed to send CNI request: Post "http://dummy/": dial unix /var/run/ovn-kubernetes/cni//ovn-cni-server.sock: connect: connection refused
2024-02-26T19:42:21.3038897Z   Feb 26 19:42:21.301: INFO: At 2024-02-26 19:41:41 +0000 UTC - event for e2e-src-ping-pod: {kubelet ovn-worker} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
2024-02-26T19:42:21.3040897Z   Feb 26 19:42:21.303: INFO: POD               NODE         PHASE    GRACE  CONDITIONS
2024-02-26T19:42:21.3044878Z   Feb 26 19:42:21.303: INFO: e2e-dst-ping-pod  ovn-worker2  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2024-02-26 19:41:36 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2024-02-26 19:41:39 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2024-02-26 19:41:39 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-02-26 19:41:36 +0000 UTC  }]
2024-02-26T19:42:21.3052505Z   Feb 26 19:42:21.303: INFO: e2e-src-ping-pod  ovn-worker   Pending         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2024-02-26 19:41:40 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-02-26 19:41:40 +0000 UTC ContainersNotReady containers with unready status: [e2e-src-ping-pod-container]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-02-26 19:41:40 +0000 UTC ContainersNotReady containers with unready status: [e2e-src-ping-pod-container]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-02-26 19:41:40 +0000 UTC  }]
2024-02-26T19:42:21.3055961Z   Feb 26 19:42:21.303: INFO: 
2024-02-26T19:42:21.3117718Z   Feb 26 19:42:21.311: INFO: Unable to fetch inter-node-e2e-196/e2e-src-ping-pod/e2e-src-ping-pod-container logs: the server rejected our request for an unknown reason (get pods e2e-src-ping-pod)
2024-02-26T19:42:21.3142158Z   Feb 26 19:42:21.314: INFO: 
2024-02-26T19:42:21.3143179Z   Logging node info for node ovn-control-plane
2024-02-26T19:42:21.3235101Z   Feb 26 19:42:21.316: INFO: Node Info: &Node{ObjectMeta:{ovn-control-plane    cdae7697-62a4-4dfd-90dd-797d75813028 14310 0 2024-02-26 17:57:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux ingress-ready:true k8s.ovn.org/ovnkube-db:true k8s.ovn.org/zone-name:ovn-control-plane kubernetes.io/arch:amd64 kubernetes.io/hostname:ovn-control-plane kubernetes.
github-actions[bot] commented 4 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] commented 2 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] commented 2 months ago

This issue was closed because it has been stalled for 5 days with no activity.