Weave GitOps provides insights into your application deployments, and makes continuous delivery with GitOps easier to adopt and scale across your teams.
Question
How should we manage Kustomizations for automation resources?
Short description
Remove currently only works in auto-merge mode and that may be a hard constraint until we start storing automation GOATs in the config repository because flux currently supports garbage collection in two ways:
Some resources (e.g. Kustomizations) can have Spec.Prune set to true. If so, and they are removed from a repository, the resources will be deleted from the cluster on the next sync.
Resources created by a Kustomization or HelmRelease are deleted if the Kustomization or HelmRelease is deleted from the cluster.
Sources do not support Spec.Prune but will be deleted if they were created by a Kustomization or HelmRelease which gets deleted.
In the user repo case, the automatic garbage collection would work for an app at path "." because the Kustomization for the app points at "." which contains the .wego directory. So, removing that Kustomization will cause the app, source and Kustomization or HelmRelease to be deleted (as they are "in the scope" of the Kustomization). In the case of an app in a subdirectory, this will fail because the Kustomization for the app will not include the .wego directory.
The same thing holds for external repositories. The Kustomization that manages the application's GOAT (app, source and Kustomize/HelmRelease) is deployed directly to the cluster so we can't remove it via a PR. Removing the automation Kustomization will cause the app's GOAT to be deleted but that means explicitly removing it from the cluster.
(Some of the above discussion is now a bit suspect -- part of it may have been due to a misunderstanding about how flux worked; however, the end result is the same; until we store the automation automation in git we won't be able to remove via PR.)
In the auto-merge case, we can go ahead and remove the automation Kustomization from the cluster and everything will work but in the PR case, it appears we have no way to make this work. If we remove the automation Kustomization up front, then when the PR is merged, it won't be synced and we won't get rid of the app GOAT or resources. If we do the PR first, we have no opportunity to remove the automation GOAT from the cluster (unless we add a controller that can clean up after the PR) as the source will not be removed from the cluster just because it was removed from the repository.
Once we store the automation GOAT in a repository or add a wego controller, we can use a PR to remove the automation GOAT and the resources it controls. Until then, though, I believe we're stuck with auto-merge for remove (although I will check with the flux team to see if I'm missing something).
Exactly how we store and manage automation GOATs depends on the platform config directory structure currently being defined by Kevin M, Mark E. and Paul F.
Timebox
This can be defined once we see where the directory structure discussions land.
Outcome
Writeup your findings here. Please include the following (if any have been created) :
jpellizzari
commented
3 years ago
Question How should we manage Kustomizations for automation resources?
Short description Remove currently only works in
auto-mergemode and that may be a hard constraint until we start storing automation GOATs in the config repository because flux currently supports garbage collection in two ways:Spec.Pruneset to true. If so, and they are removed from a repository, the resources will be deleted from the cluster on the next sync.Sources do not support
Spec.Prunebut will be deleted if they were created by a Kustomization or HelmRelease which gets deleted.In the user repo case, the automatic garbage collection would work for an app at path "." because the Kustomization for the app points at "." which contains the
.wegodirectory. So, removing that Kustomization will cause the app, source and Kustomization or HelmRelease to be deleted (as they are "in the scope" of the Kustomization). In the case of an app in a subdirectory, this will fail because the Kustomization for the app will not include the.wegodirectory.The same thing holds for external repositories. The Kustomization that manages the application's GOAT (app, source and Kustomize/HelmRelease) is deployed directly to the cluster so we can't remove it via a PR. Removing the automation Kustomization will cause the app's GOAT to be deleted but that means explicitly removing it from the cluster.
(Some of the above discussion is now a bit suspect -- part of it may have been due to a misunderstanding about how flux worked; however, the end result is the same; until we store the automation automation in git we won't be able to remove via PR.)
In the
auto-mergecase, we can go ahead and remove the automation Kustomization from the cluster and everything will work but in the PR case, it appears we have no way to make this work. If we remove the automation Kustomization up front, then when the PR is merged, it won't be synced and we won't get rid of the app GOAT or resources. If we do the PR first, we have no opportunity to remove the automation GOAT from the cluster (unless we add a controller that can clean up after the PR) as the source will not be removed from the cluster just because it was removed from the repository.Once we store the automation GOAT in a repository or add a wego controller, we can use a PR to remove the automation GOAT and the resources it controls. Until then, though, I believe we're stuck with
auto-mergefor remove (although I will check with the flux team to see if I'm missing something).Exactly how we store and manage automation GOATs depends on the platform config directory structure currently being defined by Kevin M, Mark E. and Paul F.
Timebox This can be defined once we see where the directory structure discussions land.
Outcome Writeup your findings here. Please include the following (if any have been created) :