search

kubernetes-sigs / cluster-api-provider-vsphere

Apache License 2.0
363 stars 291 forks source link

clusterclass-changes e2e test leaves behind VMs #1795

Open srm09 opened 1 year ago

srm09 commented 1 year ago

/kind bug

What steps did you take and what happened: Even after the clusterclass changes e2e test passes, the VMs are left behind in the vCenter which needs manual cleaning up on a periodic basis. Instead we would want the test to clean all the resources it creates.

What did you expect to happen: All the resources created by the test are cleaned up after the test run finishes.

Anything else you would like to add: This happens because the VSphereVM object which is creating the virtual machine on vCenter gets deleted before the controller has a chance to set the finalizer on it. This issue could be slightly mitigated by having the controller set the finalizer on the object before moving ahead with the creation of any resources. For example, the logic in the CAPI machine controller should mitigate the issue.

But even with the above logic, there is a chance for resources to be orphaned since there is some time gap between the resources getting created and the finalizers being set. Creating a CAPI issue for this one as well.

Environment:

srm09 commented 1 year ago

/assign

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

sbueringer commented 1 year ago

This is maybe fixed by https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/issues/2039

Would be nice to check if the issue is gone once #2039 is implemented and cherry-picked

sbueringer commented 1 year ago

/assign

keeping an eye on it

k8s-triage-robot commented 7 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 7 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/issues/1795#issuecomment-1901831683): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
sbueringer commented 7 months ago

/reopen /remove-lifecycle rotten

k8s-ci-robot commented 7 months ago

@sbueringer: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/issues/1795#issuecomment-1903777257): >/reopen >/remove-lifecycle rotten Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
k8s-triage-robot commented 4 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

chrischdi commented 3 months ago

@sbueringer should we close this now that we have the janitor?

sbueringer commented 3 months ago

Not sure. I think the main point here should not be that the VMs are cleaned up, but that the VMs are leaked. This could be a problem with our test or our controller (assuming it still happens)

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 2 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/issues/1795#issuecomment-2180891121): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
sbueringer commented 2 months ago

/reopen

Would be still good to potentially do something in that area.

Just an idea: what if we have some logic after all tests are run that checks if there are still some VMs and then fails the test? (independent of that we can continue to clean them up afterwards via boskosctl release)

I just think it could be good to have some additional validation if we are leaking VMs in some edge cases

(so I'm not concerned about that we "pollute" our test env with VMs, I'm more concerned about our production code leaking VMs in some cases)

(IIRC we are doing something like this in core CAPI with CAPD)

k8s-ci-robot commented 2 months ago

@sbueringer: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/issues/1795#issuecomment-2182113224): >/reopen > >Would be still good to potentially do something in that area. > >Just an idea: what if we have some logic after all tests are run that checks if there are still some VMs and then fails the test? >(independent of that we can continue to clean them up afterwards via `boskosctl release`) > >I just think it could be good to have some additional validation if we are leaking VMs in some edge cases > > > Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
k8s-triage-robot commented 4 weeks ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 4 weeks ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/issues/1795#issuecomment-2262296450): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
sbueringer commented 4 weeks ago

/reopen

/lifecycle frozen

k8s-ci-robot commented 4 weeks ago

@sbueringer: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/issues/1795#issuecomment-2262303580): >/reopen > >/lifecycle frozen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.