bookingcom / shipper

Kubernetes native multi-cluster canary or blue-green rollouts using Helm
Apache License 2.0
733 stars 39 forks source link

Change garbage collection of objects #278

Closed juliogreff closed 4 years ago

juliogreff commented 4 years ago

Currently, Shipper sets up garbage collection of objects installed by the installation controller by creating an “anchor” ConfigMap in each application cluster, and setting it as the owner of each object to be installed (through OwnerReferences). This means that once that ConfigMap is delete, an installation is garbage collected automatically by kubernetes. Additionally, the janitor controller watches InstallationTargets and the anchor ConfigMaps, and once there’s no longer an InstallationTarget for the corresponding anchor, the anchor gets deleted. This ensures that a deleted InstallationTarget (and by extension a deleted Release) has no dangling objects in application clusters.

Now that an InstallationTarget lives side by side with the objects it installs, we no longer need the ConfigMap: we can set the objects’ OwnerReferences to point to the InstallationTarget itself.

This exposes another interesting problem though: since Installation, Capacity and Traffic target objects no longer live side by side with their respective release, we can’t use OwnerReferences anymore. This means we still need the janitor controller, but it needs to collect the target objects whenever there’s no longer a Release that points to them. That should trigger the garbage collection of installed objects as well, and the clean up should be complete. This also implies that the janitor controller should live in the management cluster, in case we didn't have it in shipper-mgmt before.