cloudposse / terraform-aws-tfstate-backend

Terraform module that provision an S3 bucket to store the `terraform.tfstate` file and a DynamoDB table to lock the state file to prevent concurrent modifications and state corruption.
https://cloudposse.com/accelerate
Apache License 2.0
408 stars 177 forks source link

terraform destroy needs explanation #44

Closed schollii closed 4 years ago

schollii commented 4 years ago

Found a bug? Maybe our Slack Community can help.

Slack Community

Describe the Bug

The module docs should explain how to cleanup.

Expected Behavior

The module docs would say something like (I'm still confirming the details but I just don't want to loose this issue report):

Destroy

  1. comment out the "backend" block
  2. move the state storage back to local: terraform init
  3. make the state bucket deletable even if there are multiple versions of state stored: add force_destroy=true to your terraform_state_backend then terraform apply
  4. terraform destroy

Warning: while the state is local, the state in the bucket still exists, others (or CI/CD!) should not modify it.

osterman commented 4 years ago

@schollii you're right, this should be documented somewhere. Right now it's not.

Please have a look here for some of the scripts we've used in the past: https://github.com/cloudposse/terraform-root-modules/tree/master/aws/tfstate-backend/scripts

schollii commented 4 years ago

Thanks that's useful to know. It could be useful to mention the scripts in the docs, but also the procedure I mentioned, in case the scripts are not adequate for whatever reason.

jmcgeheeiv commented 4 years ago

@schollii @osterman. I found and documented a smooth way to apply and destroy using this module. See PR #46.

schollii commented 4 years ago

So the trick of using the module's support for automatically generating a backend.tf file works really nicely and simplifies the destruction. Here is what I use.

Creation

# main.tf
module "terraform_state_backend" {
  source                             = "git::https://github.com/cloudposse/terraform-aws-tfstate-backend.git?ref=..."
  ...
  terraform_backend_config_file_name = "backend.tf"
  terraform_backend_config_file_path = "."
  force_destroy                      = false
}

Then this is needed only once:

  1. terraform init: downloads modules and providers
  2. terraform apply -auto-approve: creates the backend.tf and the s3 bucket etc but state is stored locally
  3. terraform init -force-copy: moves the state to s3 bucket create

Since this is only once I put all these in a script init.sh that gets run once and that's it. It seems idempotent but I have not pushed testing of that very much.

Once the above is done you can define more resources and terraform apply as needed.

Deletion

  1. in main.tf edit the module.terraform_state_backend to have force_destroy true and terraform_backend_config_file_path to ""
  2. terraform apply -target module.terraform_state_backend -auto-approve: activate the above mods (s3 becomes destroyable, remove the backend.tf file)
  3. terraform init -force-copy: move the state to local host

I put all the above in a destroy.sh script that also check before step 3 that step 2 has been done (actually I could surely have the scrip do step 2 automatically but this way it is a more "conscious" decision by the user to destroy - running it by mistake will fail). Additionally the script does terraform init before step 2 because there are some failure conditions that need it.

jmcgeheeiv commented 4 years ago

Excellent. Thank you. Could you put your scripts in a comment for reference? Then I'll rewrite my doc.

jmcgeheeiv commented 4 years ago

I think this cam be closed. Nice working with you, @schollii.

schollii commented 4 years ago

likewise @jmcgeheeiv thanks for contributing the solution in #46