remote state not passing config details

[b3nnb]

I am working on upgrading to terraform 0.12.20 and terragrunt 0.21.10 from tf 0.11.14 and tg 0.18.7 Ran the terraform 0.12upgrade and it said it was successful Im also confused about when to use " and whjen not to. i see a lot of exmples that do have it and some that don't.

but the valuse of my state config do not seem to pass through. im constently getting errors like

Error: Invalid AWS Region: var.tfstate_global_bucket_region

or the name doesnt exist or cannot be found

I used to use to have 3 levels of the terragrunt tfvars. the project details the environment and the stack details

├── dev
│   ├── stack-server
│   │   └── terraform.tfvars
│   ├── common
│   │   └── terraform.tfvars
│   └── environment.tfvars
├── prod
│   ├── common
│   │   └── terraform.tfvars
│   └── environment.tfvars
├── project.tfvars
└── terraform.tfvars

i have since changes the tfvars to hcl but it only supports 2 levels the projects and the stack

├── dev
│   ├── stack-server
│   │   └── terragrunt.hcl
│   ├── common
│       └── terraform.tfvars
│   
├── prod
│   ├── common
│   │   └── terraform.tfvars
└── terragrunt.hcl

my root terragrunt looks like this

  remote_state {
    backend = "s3"

    config = {
      bucket  = "terraform-stack-server-live"
      key  = "${path_relative_to_include()}/terraform.tfstate"
      region  = "us-west-2"
      encrypt = true

      s3_bucket_tags = {
        owner = "bizops"
        name  = "Terraform state storage"
        control = "terraform-stack-server-live"
      }

      dynamodb_table_tags = {
        owner = "bizops"
        name  = "Terraform lock table"
        control = "terraform-stack-server-live"
      }
    }
  }

  terraform {

    extra_arguments "conditional_vars" {
      commands = get_terraform_commands_that_need_vars()

      // required_var_files = [
      //   get_parent_terragrunt_dir()/project.tfvars,
      //   get_terragrunt_dir()/../environment.tfvars
      // ]
    }
    # arguments = {
    #   -var-file="${get_parent_terragrunt_dir()}/project.tfvars"
    #   -var-file="${get_terragrunt_dir()}/../environment.tfvars"
    # }
  }

  inputs = {
    keyname = "stack-server-terraform"
    project = "stack-server"
    region = "us-west-2" 
    zone_name = "global.corp.com"
    tfstate_global_bucket = "terraform-stack-server-live"
    tfstate_global_bucket_region = "us-west-2"
    environment = "basename(get_terragrunt_dir())"
  }

and the stack(child) terragrunt looks like this

include {
    path = find_in_parent_folders()
  }

dependencies {
    paths = ["../common"]
  }

terraform {

    source = "git::ssh://git@github.com/terraform.git//stack-server"

}

inputs = {
  stack = "basename(get_terragrunt_dir())"
  terragrunt_remote_common_state_key = "format(path_relative_to_include()/terraform.tfstate)"
}

[yorinasub17]

Looking at your code, I believe a lot of your issues are coming from:

Im also confused about when to use " and whjen not to. i see a lot of exmples that do have it and some that don't.

In HCL2 (which is what terraform 12 and terragrunt 19+ uses), you can now have bare expressions that are types. For example, in tf11 you always had to reference variables as follows:

list_input = "${var.some_list_value}"

This is because HCL1 did not support bare expressions, and everything had to go through the string interpolation. In HCL2, you can still do what you have above, but it is preferred to use bare expressions, like so:

list_input = var.some_list_value

In general, whenever you are using an expression (e.g a function call or variable reference), you should not use quotes, unless you are creating a string with those values. For example, you can't do:

name = var.region + "some_name"

Instead, you have to construct the string using interpolation:

name = "${var.region}some_name"

Here is also a useful list of gotchas for upgrading to terraform 12

[b3nnb]

Right, and that's what i thought but when i remove the quotes from the variable calls, i just get that it cannot find the the backend.

Error: Unable to find remote state

  on data.tf line 11, in data "terraform_remote_state" "common":
  11: data "terraform_remote_state" "common" {

No stored state was found for the given workspace in the given backend.

[yorinasub17]

It's a bit hard to debug without seeing any of the relevant code here, but there are two potential issues here:

• Did you uncomment the commented out tfvars blocks in your root terragrunt.hcl config?

• Is config set as an attribute or block on the terraform_remote_state? E.g do you have:

  data "terraform_remote_state" "kms" { backend = "s3" config { // attrs omitted for brevity } }

  or do you have:

  data "terraform_remote_state" "kms" { backend = "s3" config = { // attrs omitted for brevity } }

  In tf12, the latter is correct, with the =. If you don't have that, then you actually get a silent failure where you are calling terraform_remote_state with no configuration, and thus you see the error message "no stored state found."

[b3nnb]

I did not uncomment the

//   "-var-file=${get_parent_terragrunt_dir()}/project.tfvars",
//   "-var-file=${get_terragrunt_dir()}/../environment.tfvars"

as for the data call in my data.tf file

terraform {
  backend "s3" {
  }
}

provider "aws" {
  region = local.region
}

data "terraform_remote_state" "common" {
  backend = "s3"

  config = {
    bucket         = var.tfstate_global_bucket
    key            = var.terragrunt_remote_common_state_key
    region         = var.tfstate_global_bucket_region
    encrypt        = true
    dynamodb_table = "terraform_locks"
  }
}

im happy to post anything that will help, just let me know what is needed to troubleshoot

[yorinasub17]

Are you still using quotes in terragrunt.hcl for the inputs? It should be:

inputs = {
  stack = basename(get_terragrunt_dir())
  terragrunt_remote_common_state_key = "${path_relative_to_include()}/terraform.tfstate"
}

[b3nnb]

yes i still have the quotes on the inputs. i get syntax errors if they are removed

[yorinasub17]

Can you try the version I have given you? The syntax error is caused by the fact that you are mixing an HCL expression (path_relative_to_include()) with a literal string (/terraform.tfstate). To do that, you need to use the string interpolation form (${}), as I have in my response.

[b3nnb]

Sure, that is the nested hcl file right?

so i tried it there and get

Error: Unable to find remote state

  on data.tf line 11, in data "terraform_remote_state" "common":
  11: data "terraform_remote_state" "common" {

No stored state was found for the given workspace in the given backend.

is there a way to have the path of the state file, that its looking for, output so i can see what might be incorrect?

[yorinasub17]

is there a way to have the path of the state file, that its looking for, output so i can see what might be incorrect?

So unfortunately there isn't an easy way to do this. I would suggest creating a new folder with the following:

terragrunt.hcl

include {
  path = find_in_parent_folders()
}

inputs = {
  terragrunt_remote_common_state_key = "${path_relative_to_include()}/terraform.tfstate"
}

main.tf

variable "terragrunt_remote_common_state_key" {}
output "foo" { value = var.terragrunt_remote_common_state_key }

---

Looking again, I think you want that input in the parent. path_relative_to_include only works in the included file.

[b3nnb]

i made that folder to test and the output was dev/test_state_output/terraform.tfstate

so its not looking in common at all

[yorinasub17]

That makes sense because you are seeing the relative path in the context of the test module, not the real module. This at least indicates that the config I provided will give you the right relative path to the state file.

So if you have the exact same input var config, then it should be dev/common/terraform.tfstate for the tg module running from common.

Here are a few more things to check:

• Double check the remote state block bucket name, region, and key values look sane using the same method I proposed above.
• Make sure the state file exists in S3.
• Make sure the state file has been updated to tf12. Terraform isn't really good at providing good error messages for terraform_remote_state, so this could be the issue.
• Try removing encrypt and dynamodb_table from the config. These aren't necessary when you are reading the state.
• You can also try running in debug mode TF_LOG=DEBUG terragrunt plan to see if you can gather more info.

[b3nnb]

alright, little update. with the change you gave me for my HCL file, and then me un commenting

"-var-file=${get_parent_terragrunt_dir()}/project.tfvars",

i got past that, im getting another error for a data lookup. that im sure i can sort out.

but, if im doing everything correctly, should i even need thos var files to lookup the common state data?

the tfvars has

keyname = "stack-terraform"

project = "stack"

region = "us-west-2"

zone_name = "stack-server.com"

tfstate_global_bucket = "terraform-stack-server-live"

tfstate_global_bucket_region = "us-west-2"

[b3nnb]

ahhh, i may have gotten it.

since i used 3 levels of tfvars previously.

├── dev
│   ├── stack-server
│   │   └── terraform.tfvars
│   ├── common
│   │   └── terraform.tfvars
│   └── environment.tfvars
├── prod
│   ├── common
│   │   └── terraform.tfvars
│   └── environment.tfvars
├── project.tfvars
└── terraform.tfvars

the common key was called at the environemtn level in the folder structure, so i added to the nested hcl

extra_arguments "conditional_vars" { 
    commands = get_terraform_commands_that_need_vars()   
    arguments = [
       "-var-file=${get_terragrunt_dir()}/../environment.tfvars"
     ]
  }

which it doesnt seem like there's a better way to do this since there's only allowed 2 levels of terragrunt files.

and doing this allowed me not to call the project.tfvars in the root hcl

Just have to sort out the last error that's not related to looking up the common state, and hopefully there's no more issues related to upgrading

[b3nnb]

yup, i got it. thanks for all the help and advice!!