Support of rendering to another repository

[dan1el-k]

Checklist

• [x] I've searched the issue queue to verify this is not a duplicate feature request.
• [x] I've pasted the output of kargo version, if applicable.
• [x] I've pasted logs, if applicable.

Proposed Feature

Rendering to a another repository. (Service Repo concept)

Motivation

As of version 0.3.0 kargo, just supports rendering in branches of the same repository as the source helm or kustomize files exists.

This is fine and supported for:

• Single application repos: containing manifests of one application
• Mono application repos: containing manifests of multiple application within the same repository. Those could be rendered in to a Stage branch in multiple subfolder

What I would like to have is the support of:

• Service repos: repo for the Business service, containing all the kargo deployment objects for this project, and which is used as Rendered Target and furthermore as deployment source for ArgoCD. To bundle multiple application rendered manifests in one repo, however having the original manifests still in separate repos.

With that, this Service repocould be a pure pipeline/infrastructure repo with no manual changes need after setup, and the application repos(multiple, one for each microservice) are used as a source and working repo for the team/engineers.

Suggested Implementation

Support of following new parameters

• readRepoURL: currently repoURL, mandatory
• readBranch: optional as of now
• writeBranch: optional as of now
• writeRepoURL: optional, new

Full example

apiVersion: kargo.akuity.io/v1alpha1
kind: Stage
metadata:
  name: dev
  namespace: kargo-demo-service
spec:
  subscriptions:
    warehouse: demo-kustomize
  promotionMechanisms:
    gitRepoUpdates:
    - readRepoURL: https://gh-enterprise.com/my-org/kargo-demo-kustomize
      readBranch: main
      writeRepoURL: https://gh-enterprise.com/my-org/kargo-demo-service
      writeBranch: stage/dev
      render:
        images:
        - image: artifacts.my-comp/docker/my-org/cortex-template-kustomize-application/nginx

[krancour]

Hi @dan1el-k! This is an interesting idea.

I want to try and split this into two different parts.

1. There's the more general aspect of this, where you might want to to do this for any gitRepoUpdate -- i.e. not limited to Kargo Render.

2. Then there's the Kargo Render-specific aspect of this, which is the harder one for me to wrap my head around. One of the fundamental premises of Kargo Render is that Kargo Render configuration is "close" to the source it operates on and to the branches it renders. i.e. If you were looking at repo, you would see what got rendered, what it got rendered from, and what the rules were for doing that, all in one place.

While this generally seems doable, it upsets the apple cart a little bit in terms of that fundamental premise of Kargo Render.

I think before either committing (or not committing to this), I'd like to get a sense of how common a pattern it is to read config management templates/overlays from one repo and render them to another. Let's leave this open and see where it goes.

Tangentially: idk if this other thread of yours and mine might give you an alternative way to approach this?

[dan1el-k]

In the enterprise company I am working in, teams uses different repo patterns, based on size and complexity of the applications.

• Mono repo pattern: Application source code, Kubernetes manifests, value overlays, project docu and description, pipeline settings - all in one repo. Mainly used for smaller aka 1 microservice only business services
• Mono manifest repo pattern: Application source code in separate repos per microservice, Kubernetes manifests value overlays, project docu and description, pipeline settings - all in a single manifest repo. Either used for mid-sized services to save the effort for separate manifest repos per microservice and where the team is small, or also used for service with very several similiar microservices using the same manifest with different values per microservice and stage.
• Service/Project repo pattern: Application source code in one repo with a sibling kubernetes manifest repo with value overlays for each microservice. Project docu and description, pipeline settings in a service/project repo. Mainly used for larger and/or more complex application with 3+ microservices in there application stack.

---

While the first 2 patterns are perfectly supported by kargo, I am specifically targeting the 3rd one with this request.

Yes, this could potentially be achieved as well using Kustomize remote base or an Helm umbrella chart with dependencies or via Git Submodules out of the service repo, as I have requested with my other ticket in "kargo-render" project. I do believe however, that a native support to render to a remote repo is easier to configure and grasp for the enduser

[MarkusNeuron]

Hi, this proposal is a pattern we also want to support. Besides collecting and rendering multiple repos to one service to have the deployed state of the service audited at every point in time. Taking this idea even further you could render all manifests to a cluster repo to archive the same on cluster level.

[mihaigalos]

I'm tentatively contributing a WIP PoC in https://github.com/akuity/kargo/pull/1447.

[github-actions[bot]]

This issue has been automatically marked as stale because it had no activity for 90 days. It will be closed if no activity occurs in the next 30 days but can be reopened if it becomes relevant again.

[krancour]

Not stale. I think this is something we can tentatively do, but it's a more involved change than it may seem on the surface. Priority still undetermined.

[dan1el-k]

Hey @krancour, Is there any update on this ticket ? were you able to discuss this further in the round of maintainers ? Thanks.

[krancour]

In v0.9.0, we plan to break up our "russian nesting doll" of opinionated promotion mechanisms into an ordered sequence of more granular, more explicit, user-defined steps. It is largely a guess, but this issue may be solved incidentally by those plans.

[krancour]

Following up on previous comment. I do expect that this will be addressed naturally by #2219