[Feature Idea/Suggestion] promotionTemplates as a CRD of its own?

[pww217]

Checklist

• [x] I've searched the issue queue to verify this is not a duplicate feature request.
• [ ] I've pasted the output of kargo version, if applicable.
• [ ] I've pasted logs, if applicable.

Proposed Feature

I think that it may related to this issue which I believe is about reducing the amount of total configuration and boilerplate code for promotionTemplates.

But in summary, I wonder if Kargo may be more modular and flexible if promotionTemplates were a CRD like analysisTemplates are.

Motivation

I've actually been writing a Helm Chart for my own use cases and for the most part, it's quite compact. I want to clean it up a bit more and test it, and then I intend to offer it upstream and see if there's any interest.

The chart is designed to provision a Project, Warehouses, and Stages dynamically without having to use explicit manifests or Kustomize. This idea was first floated here. The chart is meant to organize these CRs around a top-level service definition and flesh out the entire pipeline. I want it to be silly simple that my developers can manage it themselves.

Here is a sample of a values file:

owner: my-team

# Will override the naming conventions that are by default based on .Release.Name
nameOverride:

# Services definition including both warehouses and stages.
# This is design only for Github, Helm, and Amazon ECR at the moment.
services:
  - name: nginx
    imageURL: docker.io/library/nginx
    imageSelectionStrategy: NewestBuild
    git:
      repoURL: https://github.com/nginx-app.git
      valuesFilePath: infra/values # Path to values file(s) from repo root
      imageValueFile: version.yaml # The file that actually contains the image and/or tag key(s)
      key: image.tag # The key that the image is defined at in the values file.

    # Each stage represents a distinct environment or stop along the pipeline.
    # The source of each must be `direct` (from image Warehouse), or the name of a previous stage.
    stages:
      - name: dev
        autoPromotionEnabled: true
        source: direct # Pulls from the Warehouse
        argoCDAppName: nginx-dev
        verifications:
          [] # Examples of analysisTemplates below
          # - name: unit-test
      - name: staging
        autoPromotionEnabled: true
        source: dev # From previous stage
        argoCDAppName: nginx-staging
        verifications:
          []
          # - name: integration-test
          # - name: end-to-end-test
      - name: prod
        autoPromotionEnabled: false
        source: staging # From previous stage
        argoCDAppName: nginx-prod
        verifications:
          []
          # - name: canary

As you can see, if you remove the comments this will create a fully functional pipeline in only about 30 lines of values, many of which are optional.

Lately I've transition to away from promotionMechanisms to promotionTemplates, and while they were hard to figure out at first I do agree they are more flexible overall.

However, the issue comes the large amount of different ways you can configure promotionTemplates that make it very unfriendly to Helm-style templating if we want this chart to be widely flexible and usable.

Right now the chart is designed for a Helm, Github, ECR, and ArgoCD-based flow (as that's our "standard"), but I want it to be adaptable to more use cases, without everyone needing to make 100 different forks for their own org needs. But how successful that effort will be depends on how template-able it is.

Suggested Implementation

I love that with analysisTemplates we can use them like little lego blocks and just "plug them in" to verifications like:

        verifications:
          - name: integration-test
          - name: canary

You can mix and match them any way you need but the templates are defined elsewhere, independently.

The way promotionTemplates are implemented now, I can't really see how I can template it to fit all use cases without it being a very long, messy template and values file. It would be full of conditionals.

I would love it I could define promotionTemplates as separate CRs and similarly mix and match them like analysisTemplates.

That way even in a single org, if we have multiple "types" of CI flows (maybe some use ECR, some Dockerhub. Maybe some use Helm and some Kustomize), we can plug and play as needed from existing CRs in the cluster.

It's just an idea and I wonder if there's been any internal discussion this type of thing. I'll admit I have my doubts about whether it's possible, but I thought it was at least worth bringing up as an idea.

[krancour]

You should look at #2826. Judging by the timing, you and I were suggesting this at the same moment.

An interesting dimension to this is that the promotion template begins to look sufficiently generic that it could be useful in the not-too-distant-future to create a top-level PromotionTemplate CRD (and maybe ClusterPromotionTemplate, too) so Stages that all follow the same generic recipe can reference that recipe instead of repeating it.

I really want this as well, but see expression language support (coming in 1.1) as a pre-req.

[pww217]

I suppose another option I thought of was to simply place the promotionTemplate itself in the values file, but because I construct it with a bunch of other values, this wouldn't be very easy. Perhaps as a helper template, but again that feels more far hacky and a separate CRD or similar.

[pww217]

Oh interesting! Well glad it's crossed your mind, then. I'm not going to pretend I'm some veteran of writing CRDs, I'm mostly drawing a comparison between the convenience of analysisTemplates and the comparative inconvenience of promotion ones, which is especially painful when trying to construct a template like this.

I'll subscribe to that other issue and follow along. As always thanks for the quick response! And I will try to be back sooner than later to offer up this new Chart to anyone who may find it useful, if anyone at all.

[krancour]

So the promotionTemplate field on Stage (currently) is more readily compared to the podTemplate in a Deployment. It is a "template" only insofar as it is copied, in all its glory, to Promotions created for that Stage.

It is EL support that will make the promotion steps start to feel like a TEMPLATE. e.g. Fill-in-the-blanks.

I think that is what gets us to the point where a process can be defined in generic enough terms to truly be reusable, and then a CRD for it seems like a slam dunk.