search

akuity / kargo

Application lifecycle orchestration
https://kargo.akuity.io/
Apache License 2.0
1.77k stars 146 forks source link

Add Kargo Promotion Step to Remove File and/or Directory #2860

Open BenjaminNeale-Heritage opened 4 weeks ago

BenjaminNeale-Heritage commented 4 weeks ago

Checklist

Proposed Feature

The promotion directives/steps introduced in Kargo v0.9 included a 'copy' step [1]. The 'copy' step would pair well with a 'remove' step to allow individual files and/or directories to be removed. The 'git-clear' step [2] is too broad for this purpose.

Motivation

With the potential that Kargo Render will be archived in the near future, I am hoping to use Kargo to manager cluster infrastructure configuration. I currently use Kargo Render to generate Git branches for each cluster following the Rendered Manifest Pattern. Each of these Git branches contains all of the infrastructure configuration (e.g. cert-manager, Kargo, Argo CD, etc) for a single cluster. An Argo CD ApplicationSet resource on a cluster points to matching branch.

Directory structure of main branch.

├─ {COMPONENT_NAME}/
│  └─ resources/
│     ├─ base/
│     ├─ overlays/
│        ├─ {CLUSTER_NAME}/
│        └─ ...
└─ ...

Example of main branch.

├─ argo-cd/
│  └─ resources/
│     ├─ base/
│     ├─ overlays/
│        ├─ non-production/
│        ├─ production-01/
│        ├─ production-02/
│        └─ ...
├─ cert-manager/
├─ kargo/
└─ ...

Directory structure of a cluster branch (e.g. cluster/non-production branch).

├─ COMPONENT_NAME/
└─ ...

Example of cluster branch.

└─ components/
   ├─ argo-cd
   ├─ cert-manager
   ├─ kargo
   └─ ...

If Kargo is configured with a separate pipeline for each component then the 'git-clear' step available within Kargo cannot be used because all of the other components are removed from the target branch containing the rendered manifests when one of the components changes.

Here is Kargo Render config that currently achieves the same outcome.

configVersion: v1alpha1
branchConfigs:

  # =============== Non-Production Cluster ===============
  - name: cluster/non-production
    appConfigs:
      argo-cd-resources:
        configManagement:
          path: argo-cd/resources/overlays/non-production
        outputPath: components/argo-cd
      cert-manager-resources:
        configManagement:
          path: cert-manager/resources/overlays/non-production
        outputPath: components/cert-manager
      kargo-resources:
        configManagement:
          path: kargo/resources/overlays/non-production
        outputPath: components/kargo

Suggested Implementation

Removes a single file if specified.

        - uses: remove
          config:
            path: ./out/components/cert-manager/kustomization.yaml

Removes an entire directory if no file is specified.

        - uses: remove
          config:
            path: ./out/components/cert-manager

Example usage within a Kargo Stage for the cert-manager component. This would allow the cert-manager configuration to be removed without removing configuration for other components (e.g. Argo CD, Kargo, etc).

      steps:
        - uses: git-clone
          config:
            repoURL: https://github.com/example/repo.git
            checkout:
              - fromFreight: true
                fromOrigin:
                  kind: Warehouse
                  name: cert-manager
                path: ./src
              - branch: cluster/non-production
                create: true
                path: ./out
        - uses: remove
          config:
            path: ./out/components/cert-manager

References

[1] https://docs.kargo.io/references/promotion-steps/#copy [2] https://docs.kargo.io/references/promotion-steps/#git-clear

krancour commented 4 weeks ago

This is something I think we could definitely consider, but in the meantime, I think git-clear should actually work fine on any sub-directory of a working tree.

BenjaminNeale-Heritage commented 4 weeks ago

@krancour ... You are right. I didn't even think to point git-clear at a subdirectory. I can use git-clear as you've mentioned so there probably is very little need for a separate remove step.

krancour commented 3 weeks ago

@BenjaminNeale-Heritage glad it worked. I'll leave this open though, because I think the way we implemented it, git-clear won't work except within a working tree. (I need to double-check.) If that's the case, we may still require a more genetic alternative.