search

gruntwork-io / terragrunt

Terragrunt is a flexible orchestration tool that allows Infrastructure as Code written in OpenTofu/Terraform to scale.
https://terragrunt.gruntwork.io/
MIT License
8.12k stars 986 forks source link

Terragrunt apply after terragrunt init has a missing key in map #676

Open singhujjwal opened 5 years ago

singhujjwal commented 5 years ago

Hi, I am trying to use s3 as backend for terragrunt to use and it seems after the terragrunt init the terragrunt apply is missing a key in the map to identify the backend and reports that Backend has changed and creates another backend

/home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main
[ujjwal.singh@ip-10-134-199-120 vpc-main]$ terragrunt plan
[terragrunt] [/home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main] 2019/03/13 03:56:00 Running command: terraform --version
[terragrunt] 2019/03/13 03:56:00 Reading Terragrunt config file at /home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main/terraform.tfvars
[terragrunt] 2019/03/13 03:56:00 Assuming IAM role arn:aws:iam::xxxxxxx:role/xxxxx
[terragrunt] 2019/03/13 03:56:00 Terraform files in /home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main/.terragrunt-cache/WvOijy4p9anlQkkV1z6ePJqbrt0/LS7dGEQdMxmnt-Cd_Q7aUExldJQ/vpc are up to date. Will not download again.
[terragrunt] 2019/03/13 03:56:00 Copying files from /home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main into /home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main/.terragrunt-cache/WvOijy4p9anlQkkV1z6ePJqbrt0/LS7dGEQdMxmnt-Cd_Q7aUExldJQ/vpc
[terragrunt] 2019/03/13 03:56:00 Setting working directory to /home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main/.terragrunt-cache/WvOijy4p9anlQkkV1z6ePJqbrt0/LS7dGEQdMxmnt-Cd_Q7aUExldJQ/vpc
[terragrunt] 2019/03/13 03:56:00 Backend config has changed from map[key:vpc-main/terraform.tfstate region:us-east-1 workspace_key_prefix:distplat-live-df04-phoenix bucket:distplat-phoenix-live dynamodb_table:df04-phoenix-live encrypt:%!s(bool=true)] to map[key:vpc-main/terraform.tfstate region:us-east-1 encrypt:%!s(bool=true) dynamodb_table:df04-phoenix-live bucket:distplat-phoenix-live]
[terragrunt] [/home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main] 2019/03/13 03:56:00 Backend config has changed from map[bucket:distplat-phoenix-live dynamodb_table:df04-phoenix-live encrypt:%!s(bool=true) key:vpc-main/terraform.tfstate region:us-east-1 workspace_key_prefix:distplat-live-df04-phoenix] to map[bucket:distplat-phoenix-live key:vpc-main/terraform.tfstate region:us-east-1 encrypt:%!s(bool=true) dynamodb_table:df04-phoenix-live]
[terragrunt] [/home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main] 2019/03/13 03:56:00 Initializing remote state for the s3 backend
[terragrunt] [/home/ujjwal.singh/CODE/distplat/phoenix-live/df04-iedp/vpc/vpc-main] 2019/03/13 03:56:00 Running command: terraform init -backend-config=bucket=distplat-phoenix-live -backend-config=key=vpc-main/terraform.tfstate -backend-config=region=us-east-1 -backend-config=encrypt=true -backend-config=dynamodb_table=df04-phoenix-live
Initializing modules...
- module.vpc

Initializing the backend...
Backend configuration changed!

Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.

Do you want to migrate all workspaces to "s3"?
  Both the existing "s3" backend and the newly configured "s3" backend
  support workspaces. When migrating between backends, Terraform will copy
  all workspaces (with the same names). THIS WILL OVERWRITE any conflicting
  states in the destination.

  Terraform initialization doesn't currently migrate only select workspaces.
  If you want to migrate a select number of workspaces, you must manually
  pull and push those states.

  If you answer "yes", Terraform will migrate all states. If you answer
  "no", Terraform will abort.

  Enter a value: no

Migration aborted by user.
[terragrunt] 2019/03/13 03:56:04 Hit multiple errors:
exit status 1

It seems the workspace_key_prefix:distplat-live-df04-phoenix this getting lost by the terragrunt apply/plan commands. Also in s3 instead of using the workspace_prefix value it gets set to env: as LockID string in dynamoDB table distplat-phoenix-live/env:/distplat-live-df04-phoenix/vpc-main/terraform.tfstate-md5

Please let me know if i am doing anything wrong here. Thanks

singhujjwal commented 5 years ago

I am following the documentation present here for various modules https://github.com/gruntwork-io/terragrunt#interpolation-syntax

singhujjwal commented 5 years ago

The easiest way to reproduce the problem

terragrunt init -reconfigure -backend-config="workspace_key_prefix=ujjwal
terragrunt workspace new ujjwal
terragrunt apply

This throws an error saying 

Backend config has changed from map[region:us-east-1 workspace_key_prefix:ujjwal bucket:distplat-phoenix-live dynamodb_table:df04-phoenix-live encrypt:%!s(bool=true) key:vpc-main/terraform.tfstate] to map[bucket:distplat-phoenix-live key:vpc-main/terraform.tfstate region:us-east-1 encrypt:%!s(bool=true) dynamodb_table:df04-phoenix-live]

Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.

When I say yes to the prompt, i see in s3 there is a folder created with name "env:" and all the .tfstate file is present there.

My terraform.tfvars in the root directory has the below contents

terragrunt = {
  remote_state {
    backend = "s3"
    config {
      bucket         = "xxxxxxx"
      key            = "${path_relative_to_include()}/terraform.tfstate"
      region         = "us-east-1"
      encrypt        = true
      dynamodb_table = "yyyyyy"

      s3_bucket_tags {
        owner = "Ujjwal Singh"
        name  = "Terraform state storage"
      }

      dynamodb_table_tags {
        owner = "Ujjwal"
        name  = "Terraform lock for vpc"
      }
    }
  }
}

Any help is much appreciated.

brikis98 commented 5 years ago

Thanks for filing the issue and the detailed repro steps. This is the key hint as to what is happening:

Backend config has changed from map[region:us-east-1 workspace_key_prefix:ujjwal bucket:distplat-phoenix-live dynamodb_table:df04-phoenix-live encrypt:%!s(bool=true) key:vpc-main/terraform.tfstate] to map[bucket:distplat-phoenix-live key:vpc-main/terraform.tfstate region:us-east-1 encrypt:%!s(bool=true) dynamodb_table:df04-phoenix-live]

Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.

Using Terraform workspaces is modifying the config to contain a workspace_key_prefix, which Terragrunt is not handling correctly, and assumes is a diff. We've not caught this before as Terragrunt is typically used as an alternative to Terraform workspaces. A PR to fix the config check in Terragrunt to ignore workspace_key_prefix is very welcome.

singhujjwal commented 5 years ago

Thanks for taking a look at this.