Add EFS for persistent docker volumes and flexibly support creating 0+ per ECS service
Support deploying multiple databases per environment
Support flexibly deploying either an AWS Aurora postgresql or mysql database
Support specifying the correct healthcheck path for a service
Support flexibly allowing the healthcheck to be run using either curl (by default) or wget (for alpine-based images)
Support enabling AWS ECS Exec for debugging ECS tasks
Other infra:
Refactor process for updating ECS task definitions
Fix incorrect encrypted s3 IAM permission
Explicitly specify all docker compose database major versions to match AWS
Fix github actions CD concurrency string
Context for reviewers
Testing instructions, background context, more in-depth details of the implementation, and anything else you'd like to call out or ask reviewers. Explain how the changes were verified.
Some snafus that I ran into while deploying matomo to AWS:
Privileged port: The matomo image by default uses a privileged port (80), which causes issues on AWS ECS Fargate. I resolved this by building a Docker image that injects a sed command to change the apache port to 8080, but that can be adjusted with an environment variable.
Mysql: Matomo does not support postgresql (only mysql or mariadb), so this PR also changes the terraform database module to more flexibly deploy either postgres or mysql
File access: Matomo and apache need to be able to write to /var/www/html. I address this in a few ways:
Creating an EFS docker volume that maps to /var/www/html
Allowing non-read-only docker root volume
Allow the ECS task to have all EFS IAM permissions (a future @TODO would be to refactor this to a more limited scope)
General infra changes:
Retrieve image tags: I ran into a lot of headaches testing this because the terraform image_tag variable kept squashing the correct value deployed by Github with a dummy value provided in terraform. I resolved this by writing a bash script to grab the latest image tags in ECR for the participant, staff, and analytics apps and putting them in a (git ignored) image_tags.tfvars that can be used when running terraform apply
Database: This PR adds support for multiple databases per environment (one for the participant and staff portals and a separate one for analytics). It also refactors the database password into the database module
ECS service: This PR adds support for using ECS exec for debugging
Encrypted s3: Fix an incorrect IAM permission
Database version numbering: Specify the database version for all docker-compose files so that they match the versions we are using in AWS Aurora
Testing
Screenshots, GIF demos, code examples or output to help show the changes working as expected. ProTip: you can drag and drop or paste images into this textbox.
Test local development:
Uncomment line 19 in analytics/docker-compose.yml if running on an M1 mac
cd analytics && docker compose up --build
Navigate to localhost:8080 and walk through the install wizard
Test AWS deployment:
Matomo has been deployed to and installed the dev environment, so navigate to [...]() to test the deployment. Ask out-of-band for credentials
Ticket
https://wicmtdp.atlassian.net/browse/PRP-142
Changes
curl
(by default) orwget
(for alpine-based images)Context for reviewers
Some snafus that I ran into while deploying matomo to AWS:
sed
command to change the apache port to 8080, but that can be adjusted with an environment variable./var/www/html
. I address this in a few ways:/var/www/html
General infra changes:
image_tag
variable kept squashing the correct value deployed by Github with a dummy value provided in terraform. I resolved this by writing a bash script to grab the latest image tags in ECR for the participant, staff, and analytics apps and putting them in a (git ignored)image_tags.tfvars
that can be used when runningterraform apply
Testing
Test local development:
cd analytics && docker compose up --build
Test AWS deployment: