richardhallett
commented
11 months ago General thoughts:
- Version specific service modules under mono-repo
- So the module definitions for the container are versioned as one and then you reference the release of the entire datacite service for specific new modules
- This means that mastino becomes just the place where the configuration of where to deploy the modules happens.
- Define modules for common configuration like vpc/elb/service
- We should make use of more common public modules to avoid duplicated code
- This might produce a problem with how to migrate state
- We still have terraform configuration that exists in datacite/ops i.e. VPC is not configured in mastino
- When going between major versions we need to do terraform apply at least once to ensure state file is up to date with latest version
- 0.11 to 0.12 has the most syntax changes, some are not mandatory until 0.15x
- In principle getting to 0.15.x is all that is required to be able to get us to latest 1.x series due to them having non breaking compatibility in 1.x series
- in 1.1 + we can use the moved block to not recreate resources otherwise we have to manually move state see https://nedinthecloud.com/2021/12/14/using-the-moved-block-in-terraform-1-1/
Other questions with thoughts
- Why are we using terraform cloud? what do we gain?
- We can organise modules within the private registry including which public ones we recommend, but im not sure why thats useful as all we have should be public.
- State management is the big one and the automated processes associated with terraform cloud.
- Where should we be managing aws route53 dns records?
- Per service configuration as some services may not require dns
- How should we manage new versions of software releases with terraform?
- This is actually configuration management, because you're changing the software version that is used, but the actual infrastructure is staying the same.
- Task definitions in terraform shouldn't be controlled and configured here.
- Task definitions in ECS are immutable, so we should still always create a new task definition
- Instead consider to use regular aws cli to update the task definition with the new value. Also the possibility of using a tool like: https://github.com/fabfuel/ecs-deploy
- Setup dummy task definition within terraform, ignore all changes to this. via lifecycle ignore_changes
- Put the task definitions with the application code and set per environment as required. then let CI i.e. github actions do the deployment to AWS.
- https://docs.github.com/en/actions/deployment/deploying-to-your-cloud-provider/deploying-to-amazon-elastic-container-service
- How do you roll back?
- Rolling back would be the previous github tagged version because it would then update the task definition to match.
- How do we handle the environment variables set for a task configuration?
- The application specific ones should be defined in the CI tool, this is configuration management again. So we'd remove them from terraform.
Research how best to handle the upgrade process, there is existing documentation and thoughts on how this can be achieved. This issue is how best to proceed but also how to break down the work into manageable chunks.