Parameterize Terraform and Add a CI/CD Pipeline

[abdulrabbani00]

Overview

This PR contains the following:

1. A refactor of the Terraform code.
2. A CI/CD pipeline for running your terraform code.
3. A user manual.

Terraform Code

There are a few updates to the Terraform code.

1. Adding dynamic blocks - This allows users to specify multiple inbound and outbound rules, without hardcoding them into the TF code.
2. Modularizing the node_pool - By making this a module, we can reuse the code for creating the nodes. All the information about the necessary node pools can be configured in the tfvars file. This follows the concept of DRY and makes it possible to have varying numbers of node_pool based on environments.
3. Separating everything into environments. This concept is discussed further in the USER_MANUAL.md.

CI/CD Pipeline

The CI/CD pipeline will allow users to run the terraform plan, terraform apply, and terraform destroy. The setup and utilization can be found in the USER_MANUAL.md. For now its a manual pipeline that only does Terraform. A section for future work is captured in the user manual. I would love to add the following to the pipeline:

1. Integration for helmsman.
2. Python testing

Disclaimer

1. terraform destroy - The code for the terraform destroy will fail if the VPC created by Terraform is set by Digital Ocean to default. We can utilize the lifecycle block, but there are various things to consider before doing so. This is also discussed in the following thread.

Conclusion

Please read the USER_MANUAL.md which can be found at: public-merge-kintsugi/terraform/USER_MANUAL.md. This should provide a good starting point for set up and testing. Please let me know if the documentation is not clear, or if you would like further clarification. I think this is a good starting point for the IaC, and it allows for further improvements down the road.

Let me know what you think!

Best regards, Abdul Rabbani

[abdulrabbani00]

Hello @skylenet,

I hope you and the team are doing well. I wanted to know if you got a chance to test some of these changes. I would love to collaborate with you to integrate this into the codebase. Please let me know your thoughts and how you want to proceed. 😄