search

geerlingguy / ansible-for-kubernetes

Ansible and Kubernetes examples from Ansible for Kubernetes Book
https://www.ansibleforkubernetes.com
MIT License
684 stars 313 forks source link

[section_request] Tower/AWX webhooks (with GitLab) for GitOps #37

Open geerlingguy opened 4 years ago

geerlingguy commented 4 years ago

I had a note to do this elsewhere, so I figured I'd put it in the official repo. Basically, one chapter that I might or might not work on pre-1.0 is a GitOps example that uses Ansible Tower (AWX really, since I want people to be able to run it on their own without having to get a license) to build a container, test it, push it to a registry, then update the application deployment if all goes well any time a push to master happens in the GitLab repository.

This example necessarily has a lot of moving parts, so I'm kind of nervous I might not be able to build it out in CI (so it would be an example I'd have to manually integrate every now and then and update outside of just code), but it's a valuable example because there are many use cases where using Tower/AWX AAP outside of the cluster might be important (especially for integration with organizational automation management/RBAC), and a tool like ArgoCD or even GitLab's own pipelines might not be a great fit.

My idea (partially inspired by this post on GitLab + Tower + IIS deployment) is to have:

  1. A Kubernetes cluster with an app (like hello-go) deployed to it.
  2. A GitLab instance running with an access token in AWX and a repo with a webhook configured for an AWX job.
  3. An AWX instance running with a kubeconfig to be able to manage the cluster, and a job that can:
    1. Check out code from the GitLab repo.
    2. Build a new container.
    3. Run the container and run a test against it (e.g. curl).
    4. If successful, push the container image to a registry (GitLab's?)
    5. If successful, update the Kubernetes deployment with the new image.
      • (Wait for that deployment to update using the k8s module's wait parameter)
    6. Run the same test against it (e.g. curl).
carpet-stain commented 4 years ago

This is a great idea!

geerlingguy commented 4 years ago

See related: OPS BY PULL REQUEST: AN ANSIBLE GITOPS STORY

pmundt commented 3 years ago

I set out with a similar objective for experimenting with Ansible-driven GitOps within a Kubernetes cluster, but I had simplified requirements as I wasn't looking to build a full build-deploy pipeline, and instead am using playbooks in my repository to handle the installation and manipulation of Helm charts and related Kubernetes manifests into the running cluster.

It took a bit of wrangling, but I now have a .gitlab-ci.yml that configures tower-cli, synchronizes the project/inventory/job template in AWX (running in kubernetes directly), launches a job based on the template, waits for it to complete, and passes off the return result of the job to the pipeline.

One area that's still a bit question is that cluster access credentials are scoped by the ServiceAccount that AWX is installed under, and we have no straightforward way of mapping Kubernetes ServiceAccounts into AWX credentials. The Helm charts work around this by providing their own ServiceAccounts and ClusterRoles, but it still leaves things a bit more widely scoped than I would like in the default cluster context.