openstreetmap / operations

OSMF Operations Working Group issue tracking
https://operations.osmfoundation.org/
98 stars 13 forks source link

Move Terraform state from Terraform Cloud to self hosted #957

Open Firefishy opened 1 year ago

Firefishy commented 1 year ago

Current we use a free Terraform Cloud account for storing our terraform state for AWS, StatusCake and future Fastly terraform.

We should setup a "self hosted" dedicated terraform S3 bucket (state storage) + DynamoDB (locking) per https://developer.hashicorp.com/terraform/language/settings/backends/s3

Setup Guides: https://spacelift.io/blog/terraform-s3-backend https://blog.gruntwork.io/how-to-manage-terraform-state-28f5697e68fa

Risks: The S3 + DynamoDB access credentials would be stored in the terraform init.tf, this isn't an ideal solution as the state contains secrets. A better solution may need to be considered.

Firefishy commented 1 year ago

This is currently motivated by Terraform Cloud mostly being down: https://status.hashicorp.com/incidents/9t8nt37n4qg0

Firefishy commented 10 months ago

Terraform Cloud's free tier is current upto 5 users. Hashicorp changing the free tier to be based on the number of managed resources (<500 resources). The deadline to switch to the new model is by end of 2023. This forces our hand to move before the end of the year.

We'll likely switch as soon as a production version of opentofu is released.