search

microsoft / azure-container-apps

Roadmap and issues for Azure Container Apps
MIT License
370 stars 29 forks source link

Feedback: Feature Request - Pull based deployment method (GitOps style) #193

Open pelithne opened 2 years ago

pelithne commented 2 years ago

Describe the solution you'd like. The ability to use a pull based approach to deploying workloads into a container app environment. An example of pull based deployment method is GitOps.

In other words, to have a "client" running in the environment, which periodically monitors a git repository and automatically pulls and applies any changes made to a specific branch in that repo (for instance when a PR is merged into the main branch).

The Git repository will be the single source of truth, and will contain the descriptions of the desired state of the running application(s), and there is an automated process to make the environment match the described state in the repository.

In the case of Kubernetes, the client would apply any changes to a yaml-manifest (or helm chart). For a container app environment, a bicep file or ARM Template (or a collection of those) should be applied instead.

Some requirements:

Example Scenario: Easily deploy to private environment. When an environment is fully private, it becomes a challenge to deploy using tools like Github actions and Azure DevOps, since connectivity to the control plane is needed. This can be solved by opening up for (huge) ip ranges of the hosted agents used by Azure DevOps or Github, but this is rarely an acceptable solution. Another alternative is to use self hosted agents, in a VNET with access to the control plane, but this introduces operational cost and cost of ownership, as well as adding complexity.

A cleaner solution is to use a pull based deployment, from inside the container app environment. This removes the need to whitelist ip addresses, and/or the need to host virtual machines for self hosted agents.

Flux and ArgoCD are examples of GitOps tools, which are used for GitOps in Kubernetes.

Describe alternatives you've considered. The other alternative already exists, with CD pipelines. Compared to a typical CD pipeline, this method has the advantage that it operates in an "inside out" manner.

Additional context. Gitops is quickly becoming a popular solution for deploying applications and infrastructure to Kubernetes. There is reason to believe that this method of deployment will be requested for container apps as well.

denniszielke commented 2 years ago

+1 This would really improve the productivity for using multiple container apps.

marcindulak commented 2 years ago

Some more requirements:

  1. possibility to temporarily disable the sync (for example when doing manual emergency deployments), see the analogous request in https://github.com/argoproj/argo-cd/issues/4808
  2. support for a specific repository/branch/commit/folder to be deployed https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#applications
  3. UI and cli diff support, like in helm diff or kapp diff https://carvel.dev/kapp/docs/develop/diff/. See also https://github.com/fluxcd/flux2/discussions/820
  4. manual gating https://github.com/fluxcd/flux2/discussions/870#discussioncomment-420896
  5. automated promotion between environments https://github.com/argoproj/argo-cd/discussions/5667
simongottschlag commented 2 years ago

@pelithne we are looking into building something like this until there's official support. My thought is that we will have a reconciler running as a background task in a managed environment and then have a separate RG where all other apps are created. We will have to create our own definition of an app etc that can be stored in a readable way in git.

I'll ping you when (if) we have something to try out.

joelnotified commented 2 years ago

+1

We've got an awesome setup using Flux with AKS today, and we'd like to see something similar in ACA if we were to consider moving (ACA looks super promising). And aside from the benefits listed above, we're really liking the speed at which we can experiment in non production environments: Just push changes to a git repo and seconds later your changes are live. Revert a commit and you're back.

joshuadmatthews commented 2 years ago

@pelithne ACR Tasks can handle some of these requirements. They can listen for repo changes and automatically build images and push them into a network restricted registry. In my setup I am triggering the ACR tasks manually from azure pipelines, and then once the ACR task has executed I use CLI to update the container app to the new image.

This setup allows not to open up huge ip ranges and also not needing vms.

simongottschlag commented 2 years ago

Hi!

I've released an initial MVP of a GitOps engine for Azure Container Apps: https://github.com/XenitAB/azcagit

Feel free to test it out. :+1:

nwmcsween commented 1 year ago

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

calleo commented 1 year ago

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

nwmcsween commented 1 year ago

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

No. You would deploy fluxcd, install tf-controller and workload identity, create terraform ACA deployment in git, create tf-controller with workload ident creds with ref to aca git repo, a bit more complex but allows a lot more.

simongottschlag commented 1 year ago

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

No. You would deploy fluxcd, install tf-controller and workload identity, create terraform ACA deployment in git, create tf-controller with workload ident creds with ref to aca git repo, a bit more complex but allows a lot more.

Or just use azcagit šŸ˜ŠšŸ¤·ā€ā™‚ļø

ezYakaEagle442 commented 9 months ago

Hi, @denniszielke @dariagrigoriu could you please add this RFE in the ACA Roadmap ? Would it be possible to leverage the AKS underlying Flux Add-on ?

Other use case related to the Flux ecosystem:

Thank you, Steve ex GBB ;)

magohl commented 3 months ago

Anything new related to pull based GitOps for ACA?

Agree on the comments that something related to Flux would be ideal.

azcagit look great (thanks @simongottschlag) and for a small hobby project i would not hesitate to use it but for a long term commitment it does not feel ideal.

ameya-karbonhq commented 3 months ago

Any idea when this will be out or at least if MS can share Roadmap for future releases?

cveld commented 2 months ago

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

No. You would deploy fluxcd, install tf-controller and workload identity, create terraform ACA deployment in git, create tf-controller with workload ident creds with ref to aca git repo, a bit more complex but allows a lot more.

I don't fully understand. Would fluxcd with tf-controller already run in ACA as we know it today or would this require any evolution?

simonjj commented 2 months ago

@simongottschlag this is quite awesome! We're a little late in noticing, thank you very much for sharing. How do people feel about getting something like this implemented into ACA? Can you share (comment or thumbs up) if you:

simonjj commented 2 months ago

yes, it took me 5 times to get this "just right"

magohl commented 2 months ago

@simonjj something like that implemented into ACA would absolutely have me look at ACA as an alternative to AKS in more scenarios.

Tim-Hodge commented 2 months ago

Have implemented a simple yaml -> container app GitOps tool using Pulumi, but would welcome something built in or a simple way to use a popular tool like Flux.

fritzfs commented 2 months ago

Any idea when this will be out or at least if MS can share Roadmap for future releases?

This seems to be the roadmap - https://github.com/orgs/microsoft/projects/540

fritzfs commented 2 months ago

This could also be modeled using tf-controller in fluxcd and terraform, although I don't really know the pros and cons to using tf-controller vs azcagit @simongottschlag?

Wouldn't I have to install the fluxcd operator in the ACA environment for that to work? Is that even possible?

No. You would deploy fluxcd, install tf-controller and workload identity, create terraform ACA deployment in git, create tf-controller with workload ident creds with ref to aca git repo, a bit more complex but allows a lot more.

I don't fully understand. Would fluxcd with tf-controller already run in ACA as we know it today or would this require any evolution?

I don't see how this can be achieved without the Microsoft effort. They already integrated flux for Azure Acr and Azure Kubernetes Service (https://learn.microsoft.com/en-gb/azure/azure-arc/kubernetes/conceptual-gitops-flux2) so let's hope they can embed it here also ...

ameyaagashe commented 2 months ago

@fritzfs I do not see GiOps for ACA still on MS Azure Roadmap! Help me rectify if I am wrong.

ameyaagashe commented 2 months ago

How do I contribute to ACA? I want to start developing a GitOps Operator for ACA. We live in a world of modern software engineering, and it isn't very pleasant not to have GitOps in ACA.