Closed andreaskaris closed 1 month ago
I think https://github.com/ovn-org/ovn-kubernetes/commit/226a49b2a7b3a23e6e17f19ad4eeceeb256e747a might have improved this a lot.
@andreaskaris , I was starting to look at what I think is this same problem last week. jira is here: https://issues.redhat.com/browse/SDN-2323
See my comment there where I noticed some job looked like it just hung for 45m before getting cancelled.
The time out on pod creation can be observed in a lot of tests.
It's here, too: https://github.com/ovn-org/ovn-kubernetes/runs/3944199673
@jluhrsen I don't know if those are related. For the ones that I'm talking about, pod creation times out for the majority of tests of a specific test run. It happens only for a few tests though. I could reproduce this only once in my local setup, but since my laptop went into standby, I lost that cluster. From my observation last night, it seemed to be related to the CNI not being able to talk to the cluster, but without a reproducer, it's more speculation than anything.
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 5 days.
This issue was closed because it has been stalled for 5 days with no activity.
I hit this regularly in my PR https://github.com/ovn-org/ovn-kubernetes/pull/2582, but I also see this here in an unrelated (yet merged) PR: https://github.com/ovn-org/ovn-kubernetes/runs/3885062356
From the above log:
I could also reproduce this locally. Under certain circumstances, one will see the following in kubelet logs:
And this leads to a timeout of shard tests.
It times out here: go-controller/pkg/cni/cni.go: