search

fabric8io / kubernetes-client

Java client for Kubernetes & OpenShift
http://fabric8.io
Apache License 2.0
3.39k stars 1.46k forks source link

Improve test stability #4507

Closed shawkins closed 1 year ago

shawkins commented 1 year ago

Is your task related to a problem? Please describe

Both the regular and integration tests are routinely failing.

Describe the solution you'd like

Diagnose and update the test logic - from some of the recent prs:

Timeout is too short: https://github.com/fabric8io/kubernetes-client/actions/runs/3272864707/jobs/5384383421#step:4:30639 https://github.com/fabric8io/kubernetes-client/actions/runs/3275507794/jobs/5390447428#step:4:33888 Testing locally an informOnCondition can take over 300ms, 1 second on github is cutting it too close. Based upon the other test timeouts it seems like 10s is the most common in use for mock tests, so I'll update the tests to that.

Karaf artifact related: https://github.com/fabric8io/kubernetes-client/actions/runs/3272243272/jobs/5383015653#step:4:44624 https://github.com/fabric8io/kubernetes-client/actions/runs/3249472532/jobs/5331916273#step:4:51981

Delete never processed: https://github.com/fabric8io/kubernetes-client/actions/runs/3250738892/jobs/5335104309#step:6:441 https://github.com/fabric8io/kubernetes-client/actions/runs/3284978788/jobs/5411769181#step:6:456

Describe alternatives you've considered

No response

Additional context

No response

manusa commented 1 year ago

The Karaf tests were already refactored a while back, we managed to improve their stability, but with newer versions of Karaf, it seems to have degraded again.

For the timeouts, we should really try to make the tests run quicker. We already did a few iterations on this in the past. We should probably spend again some time on this. Many of the new tests should also be split into multiple, more granular cases with single assertions (this should also improve their stability).

For the delete operations, I also remember refactoring part of this not so long ago. IIRC in some cases we're performing unnecessary assertions, but I need to further check this.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!