search

akuity / kargo

Application lifecycle orchestration
https://kargo.akuity.io/
Apache License 2.0
1.64k stars 142 forks source link

discussion: ephemeral environments #51

Open krancour opened 2 years ago

krancour commented 2 years ago

Let's start talking about the use cases for this and how we might wish for these to work.

krancour commented 2 years ago

@jessesuen can you comment on what some use cases for this might be?

I know one use case you mentioning was about creating ephemeral environments to test individual PRs. The more that I think about that, the more I wonder if that isn't actually a CI problem more so than a CD problem.

I'm sure there are other use cases for this.

krancour commented 1 year ago

Coming back to this, it's still not clear to me how this fits with what we've spec'ed out as a group.

Spinning up and tearing down environments to test a PR seems a completely different problem from what we've been solving, and I think it's well addressed by other tools in the CI space.

@jessesuen do you think we can close this one? Or at minimum, deem this out of scope for an MVP?

github-actions[bot] commented 12 months ago

This issue has been automatically marked as stale because it had no activity for 90 days. It will be closed if no activity occurs in the next 30 days but can be reopened if it becomes relevant again.

highb commented 12 months ago

Coming back to this, it's still not clear to me how this fits with what we've spec'ed out as a group.

Spinning up and tearing down environments to test a PR seems a completely different problem from what we've been solving, and I think it's well addressed by other tools in the CI space.

@jessesuen do you think we can close this one? Or at minimum, deem this out of scope for an MVP?

In my experience at my company, I agree. There are dedicated tools like Garden and Skaffold that deal with all the complications of dev work that you encounter in CI much better than I think Kargo can aspire to. I think it would be better if Kargo focuses on the CD Promotion process, which will indirectly help these tools. For example, if someone has Garden configured with their new "remote Kustomize sources" feature, they can pull from a "development" or "ephemeral-environment" overlay into their Garden run, so Kargo could auto-promote into that env, as well as integration or other shared testing environments.

github-actions[bot] commented 8 months ago

This issue has been automatically marked as stale because it had no activity for 90 days. It will be closed if no activity occurs in the next 30 days but can be reopened if it becomes relevant again.

krancour commented 8 months ago

Not stale.

@jessesuen and I have had further offline discussion about this. Here's a recap.

  1. Argo CD can do most of this, but it seems few people are aware of that.

    https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/Generators-Pull-Request/

  2. A lot of people are asking for this to be integrated into Kargo. We hear that and won't ignore it.

  3. Since this facilitates testing of PR code pre-merge, we believe it falls squarely on the CI side of things.

  4. Since we want to maintain a crisp separation between CI and CD and since everything we have developed in Kargo so far is exclusively CD-oriented, we believe this functionality should be implemented as a completely separate feature set. i.e. Probably a whole separate set of CRDs and controllers with dedicated UI views.

Removing the stale label.

gerhard commented 7 months ago

I was talking to @kencochrane about this some weeks ago, and also today.

For as long as I can remember, my approach has been: "if it's not in production, it's inventory". If you believe in this, then it's just a small step to: "minimise time from dev to prod".

As for the last key principle, approaching everything as an experiment, trying to figure out as quickly as possible if it works, and always iterating on "good enough" is essential to producing high-quality software.

From my perspective, these three tenets are foundational to this "ephemeral stages" discussion:

  1. If it's not in production, it's inventory
  2. Minimise time from dev to prod
  3. Everything is an experiment - keep iterating on "good enough"

In my experience, the best way of achieving the above has a few note-worthy parts:

I will ignore "feature flags" in this conversation, and just focus on the "ephemeral, short-lived preview environments" part.

Today, we are using preview environments via Argo's ApplicationSet & the pull request generator, exactly as @krancour linked above. Our approach is rooted in 🎬 Preview Environments with ArgoCD - Brandon Phillips, Codefresh. While it works weell, the hardest part is forking production data, as well as being able to visualise all PRs in flight in a single place. Argo CD works, but it was not built for that - there is too much detail and it becomes overwhelming past a certain point. Kargo's UI has the potential of becoming much better for this purpose.

I have hit my time-box for this, but I would be happy to continue having this conversation. Next week at KubeCon EU is an option. @kencochrane has all the details.

cc @matipan @marcosnils @samalba

github-actions[bot] commented 4 months ago

This issue has been automatically marked as stale because it had no activity for 90 days. It will be closed if no activity occurs in the next 30 days but can be reopened if it becomes relevant again.

krancour commented 4 months ago

The thinking on this, for quite some time, has been that this doesn't fit neatly into our pipeline-oriented view of the world and would likely be the foundation of an entirely new and separate set of features.

So this is not stale, but is low priority while we are focused on hardening the original feature set.

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it had no activity for 90 days. It will be closed if no activity occurs in the next 30 days but can be reopened if it becomes relevant again.