Implement CI/CD Pipeline with Kubernetes and GitHub Actions

[JeremyEastham]

This issue is intended to consolidate the following:

• 96

• 132

• 175

Last semester, I deployed the current version of our project to the DigitalOcean web hosting service using a "simple" Kubernetes configuration. The version of the project deployed to the Droplet is available under the project-presentations branch. There are several issues with the current configuration:

• Deploying the project is complicated, undocumented, and requires admin access to the server
• The database and logs will be destroyed when we update the code
• No web scraping is currently being done on the server

I plan to design a pipeline that will allow us to easily deploy our system without directly accessing the server. Here is an outline of how we would update AgTern:

1. Someone suggests an idea and creates a GitHub Issue
2. Someone implements the code to create the idea and submits a PR
3. The code is merged into the dev branch in the repository
4. When we have made enough progress on the dev branch, we merge dev into main (maybe prod or something similar?)
5. A GitHub Actions Runner Controller on the server is watching the main branch for changes, sees the update, and executes a GitHub Action with a Self-Hosted Runner
6. The GitHub Action will build the code into a Docker image on the server and automatically deploy it according to the kubernetes.yaml configuration file in the root of the repository

To-Do

• [x] Install prerequisites
  • [x] Install Docker
  • [x] Install MiniKube for local testing of kubernetes.yaml
  • [x] Install the Helm Package Manager CLI for Kubernetes
• [ ] Follow the instructions to set up the GitHub Actions Runner Controller on the server
• [ ] Write a GitHub Action to clone the repo, build a Docker image, and apply the kubernetes.yaml configuration file to the server when we push to main (kubectl apply -f kubernetes.yaml --prune --all or similar)
• [ ] Write a kubernetes.yaml file that will create Deployments for both the GitHub Actions Runner Controller and the newly built AgTern image, including persisting the logging and data directories with PersistentVolumes and PersistentVolumeClaims

This setup will take a lot of work initially, but my hope is that it will "just work" once it is complete so that we can focus on building AgTern!

[JeremyEastham]

I have decided that it will greatly simplify this process to convert AgTern into a Helm Chart. This allows us to depend on other Helm Charts such as Postgres, the GitHub Actions Runner Controller, and the Kubernetes Dashboard. I'm updating project-presentations with these changes.