search

kubeshop / testkube

☸️ Kubernetes-native testing framework for test execution and orchestration
https://testkube.io
Other
1.31k stars 130 forks source link

Lost connection Testkube <> queued Job in quoted namespace - no proactive pod watching #5076

Open loorcker opened 6 months ago

loorcker commented 6 months ago

Describe the bug When launching two tests simultaneously in a namespace with ResourceQuota, Teskube looses contact with the test/job that waits for resources.

To Reproduce Steps to reproduce the behavior: Best to observe jobs and pods directly on k8s during test execution:

  1. Set resource quota limits to run only 1 job at a time.
  2. Run two tests within 3 seconds, allowing one to use all available resources.
  3. Watch the info on the testkube dashboard on the second test.
  4. Once the resources are freed by test no 1, test no 2 runs underneath - Teskube doesn't know about it.

Expected behavior Testkube ensures that it maintains contact with "queued" jobs and proactively monitors the pods.

Version / Cluster Kubernetes: v1.20.2 Testkube: API Version v1.16.26 Dashboard Version v1.16.5 Helm Chart Version v1.16.34

Screenshots

  1. On dashboard clicked twice "Run now" image

  2. Test number 1 is currently being processed. Test number 2 is waiting for available resources. image image

  3. The test number 1has been completed. Test number 2 is currently being processed image image

  4. Test number two stays with an Error from the beginning, but is still being processed underneath: image

vsukhin commented 6 months ago

thank you @loorcker definitely we need to improve it