Recreating ResourceComposition instance

devdattakulkarni commented 1 year ago

We have observed that the following sequence of events is possible:

Create ResourceComposition instance -> This leads to the creation of the application CRD
KubePlus Pod gets deleted from the cluster
ResourceComposition instance gets deleted -> This leaves behind the application CRD and its instances (since KubePlus is not around, deletion of ResourceComposition instance does not lead to delete of application CRDs and their instances)
KubePlus is installed
Trying to create a ResourceComposition instance to "acquire/subsume" the existing application CRD and its instances fails with the following type of error:

Error from server: error when creating "Jenkins.yaml": admission webhook "platform-as-code.crd-binding.v1" denied the request: Resource with Kind Name Jenkins exists in the cluster.

devdattakulkarni commented 1 year ago

@eminalparslan Can you take a look at this issue?

devdattakulkarni commented 1 year ago

@eminalparslan

Kubernetes finalizers will be an appropriate mechanism to handle this situation: https://kubernetes.io/docs/concepts/overview/working-with-objects/finalizers/

The general idea will be like this: When KubePlus Pod is about to be deleted, we add metadata.finalizers entry to all the resourcecomposition instances that are currently present in the cluster. This will prevent Kubernetes API to delete resourcecomposition instances. When KubePlus starts up, it will check for any existing resourcecomposition instances with metadata.finalizers field. If it finds such resources, KubePlus can simply clear the metadata.finalizers entry. This will cause Kubernetes API to proceed with the deletion of that resourcecomposition if one was requested while KubePlus was down. And since KubePlus is now back, the resourcecomposition deletion should naturally lead to deletion of all the children Custom Resource instances. (This assumes that KubePlus, upon startup, is able to build its internal state correctly by discovering any resourcecomposition instances that are currently present in the cluster. This functionality is present in KubePlus, though it will be good to verify it).

devdattakulkarni commented 7 months ago

Pre-requisites:

Understanding of KubePlus architecture
Understanding of Kubernetes finalizers
Golang

devdattakulkarni commented 3 months ago

This is no longer an issuer after moving resourcecomposition crd registration in the KubePlus helm chart's crd folder. See: https://github.com/cloud-ark/kubeplus/commit/34f6dcef47dd00b7fd3c2a42c3e046fab33c6f42

cloud-ark / kubeplus

Recreating ResourceComposition instance #1147