kubevirt / kubevirt

Kubernetes Virtualization API and runtime in order to define and manage virtual machines.
https://kubevirt.io
Apache License 2.0
5.65k stars 1.35k forks source link

CI silently skips tests requiring certain storage classes #9804

Open 0xFelix opened 1 year ago

0xFelix commented 1 year ago

What happened:

Helpers like libstorage.CreateFSPVC or libstorage.CreateBlockPVC search for a suitable storage classes before creating a PVC. If no storage class was found Skip is called and the test case is skipped. This might come as a surprise to test case authors as it is not obvious test cases might be skipped when using those helpers.

Several instance type test cases in lane sig-compute are affected, because no block storage is available there.

See e.g. https://storage.googleapis.com/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/9799/pull-kubevirt-e2e-k8s-1.25-sig-compute/1661927631949402112/artifacts/junit.functest.xml and search for skipped.

What you expected to happen:

Tests cases should not be silently skipped when a storage class is missing.

dhiller commented 1 year ago

IMHO tests cases should never be silently skipped. Thanks for bringing this up, @0xFelix !

dhiller commented 1 year ago

In general tests should either be skipped explicitly or fail.

xpivarc commented 1 year ago

In general tests should either be skipped explicitly or fail.

100% agree!

kubevirt-bot commented 1 year ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

kubevirt-bot commented 1 year ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

0xFelix commented 1 year ago

/remove-lifecycle rotten

kubevirt-bot commented 10 months ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

kubevirt-bot commented 9 months ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

0xFelix commented 9 months ago

/remove-lifecycle rotten

kubevirt-bot commented 6 months ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

kubevirt-bot commented 5 months ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

kubevirt-bot commented 4 months ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

/close

kubevirt-bot commented 4 months ago

@kubevirt-bot: Closing this issue.

In response to [this](https://github.com/kubevirt/kubevirt/issues/9804#issuecomment-2241253401): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
dhiller commented 4 months ago

/reopen /lifecycle frozen

kubevirt-bot commented 4 months ago

@dhiller: Reopened this issue.

In response to [this](https://github.com/kubevirt/kubevirt/issues/9804#issuecomment-2242986714): >/reopen >/lifecycle frozen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.