search

stefanprodan / timoni

Timoni is a package manager for Kubernetes, powered by CUE and inspired by Helm.
https://timoni.sh
Apache License 2.0
1.53k stars 68 forks source link

[Proposal] Multi-step delete #351

Open robinkb opened 7 months ago

robinkb commented 7 months ago

Hello!

First of all, thank you for developing Timoni! It's already a wonderful tool, and I hope to see it grow further. On to business.

I am building a Kubernetes cluster using Timoni with Flux. Timoni is used to install Flux, and a set of OCIRepository and Kustomize resources. The OCI artifacts are Kubernetes manifests built with timoni build. The creation of the Kustomize resources is ordered using Timoni's multi-step apply. Components are brought up in an ordered manner without errors.

Unfortunately, tearing down the cluster is not straight-forward. Multi-step delete would make this easier, where resources are deleted in the reverse order defined in the multi-step apply.

If you are open to this suggestion, I could have a crack at developing the feature based on the multi-step apply functionality.

Other solutions that I've checked:

  1. Flux Kustomization's spec.dependsOn. Unfortunately, defined dependencies are not honored during deletion. This applies to the Helm controller as well as the Kustomize controller. These issues are quite old at this point, and it appears to be a complicated problem in Flux because of multi-tenancy reasons.
  2. I see that Bundles do delete instances in reverse. Perhaps I should be using bundles to orchestrate my cluster instead? Even so, I think that multi-step destroy would still be useful to have on the module level.
stefanprodan commented 7 months ago

Perhaps I should be using bundles to orchestrate my cluster instead?

Yes that's how I'm managing the Flux AIO setup and the various modules available here: https://timoni.sh/flux-aio/

robinkb commented 7 months ago

Thanks for the quick response! I will probably rework my setup to use Bundles then.

What about the proposal though? It might not be the best solution for my particular use-case, but perhaps the feature would be valuable for other situations. If not, I can close the issue.

stefanprodan commented 7 months ago

Deletion is done in the reverse order based on kind weight, namespaced first and global last, but it’s done in background without waiting on finalizer. The issue is that whatever is in the module multi-step is not in-cluster, that information is only available at install/upgrade as deletion does not require for the module to exist. Imagine if deletion would fail if the module in the registry would not be available.

robinkb commented 7 months ago

Ah yes, that would be a problem. The solution for that would be to persist the multi-step information in-cluster in the Instance Secret.