Terragrunt does not mix parent terragrunt.hcl content into .terragrunt-cache

[xyklex]

Hi, I don't know if this is a bug or maybe a misconfiguration issue, but this schema worked the last time I used terraform and I don't find any other reason why it does not works.

Versions: Terraform v0.12.3 Terragrunt v0.19.6

My directory structure

./
├── bin
│   ├── terraform
│   └── terragrunt
├── do-kubernetes-creds.json
└── live
    ├── sandbox
    │   └── terraform-instance
    │       └── terragrunt.hcl
    ├── stage
    └── terragrunt.hcl

cat ./live/sandbox/terraform-instance/terragrunt.hcl

terraform {
  source = "../../../../tf-modules//bastion-host"
}

include {
  path = find_in_parent_folders()
}

cat ./live/terragrunt.hcl

remote_state {
  backend = "gcs"
  config = {
    bucket         = "devops"
    prefix         = "terraform-state/${path_relative_to_include()}/terraform.tfstate"
  }
}

$ terragrunt plan
[terragrunt] [/Users/monkey/tf-test/live/sandbox/terraform-instance] 2019/06/27 13:03:30 Running command: terraform --version
[terragrunt] 2019/06/27 13:03:30 Reading Terragrunt config file at /Users/monkey/tf-test/live/sandbox/terraform-instance/terragrunt.hcl
[terragrunt] 2019/06/27 13:03:30 Cleaning up contents of download folder /Users/monkey/tf-test/live/sandbox/terraform-instance/.terragrunt-cache/C7klnJ0JOvtrV3YBXXBqrdYa60k/RtsJHkaVlj16ypBH01STcx8XS2k
[terragrunt] 2019/06/27 13:03:30 Downloading Terraform configurations from file:///Users/monkey/Directories/dial-once/tf-modules into /Users/monkey/tf-test/live/sandbox/terraform-instance/.terragrunt-cache/C7klnJ0JOvtrV3YBXXBqrdYa60k/RtsJHkaVlj16ypBH01STcx8XS2k
[terragrunt] 2019/06/27 13:03:30 Copying files from /Users/monkey/tf-test/live/sandbox/terraform-instance into /Users/monkey/tf-test/live/sandbox/terraform-instance/.terragrunt-cache/C7klnJ0JOvtrV3YBXXBqrdYa60k/RtsJHkaVlj16ypBH01STcx8XS2k/bastion-host
[terragrunt] 2019/06/27 13:03:30 Setting working directory to /Users/monkey/tf-test/live/sandbox/terraform-instance/.terragrunt-cache/C7klnJ0JOvtrV3YBXXBqrdYa60k/RtsJHkaVlj16ypBH01STcx8XS2k/bastion-host
[terragrunt] [/Users/monkey/tf-test/live/sandbox/terraform-instance] 2019/06/27 13:03:30 Initializing remote state for the gcs backend
[terragrunt] 2019/06/27 13:03:30 Missing required GCS remote state configuration project
[terragrunt] 2019/06/27 13:03:30 Unable to determine underlying exit code, so Terragrunt will exit with error code 1

ls .terragrunt-cache/C7klnJ0JOvtrV3YBXXBqrdYa60k/RtsJHkaVlj16ypBH01STcx8XS2k/bastion-host

main.tf        terragrunt.hcl vars.tf

cat .terragrunt-cache/C7klnJ0JOvtrV3YBXXBqrdYa60k/RtsJHkaVlj16ypBH01STcx8XS2k/bastion-host/terragrunt.hcl

terraform {
  source = "../../../../tf-modules//bastion-host"
}

include {
  path = find_in_parent_folders()
}

As you can see terragrunt does not mixed the root terragrunt.hcl remote state config file with the child one, I know this does not not represent the documentation neither terragrunt v0.18.x behavior.

Thanks in advance for the time anyone invest in looking in my issue.

[xyklex]

I found that using Terragrunt v0.19.0 neither merge the terragrunt.hcl file contents in .terragrunt-cache but now the initialization backend works. So actually something is wrong with v0.19.6

[brikis98]

@robmorgan Could this be a bug with the recently added GCP/GCS support?

[diegoitaliait]

Hi i have the same problem with Azure.

terragrunt: 0.19.6 terraform: 0.12.3

this is my tree:

.
├── README
├── terragrunt.hcl
├── vars.tfvars
└── westus2
    ├── mgmt-subnet
    │   └── terragrunt.hcl
    ├── mgmt-vm-diego-lab-01
    │   └── terragrunt.hcl
    ├── mgmt-vnet
    │   └── terragrunt.hcl
    ├── rg
    │   └── terragrunt.hcl
    └── terragrunt.hcl

This is my root terragrunt.hcl

remote_state {
  backend = "azurerm"

  config = {
    resource_group_name  = "${get_env("RG_NAME_INFRA", "diego-infra")}"
    storage_account_name = "${get_env("TERRAFORM_STORAGE_ACCOUNT_NAME", "diegotf")}"
    container_name       = "${get_env("TERRAFORM_CONTAINER_NAME", "diego-tf")}"
    key = "development/${path_relative_to_include()}/terraform.tfstate"
  }
}

This is my root/westus2/terragrunt.hcl

inputs = {
    location = "westus2"
}

And finally this is my live hcl file root/westus2/rg/terragrunt.hcl

terraform {
  source = "${find_in_parent_folders()}/../../../diego-azure-tf-modules//resource_group"
}

# Include all settings from the root terraform.tfvars file
include {
  path = find_in_parent_folders()
}

with terragrunt plan into root/westus2/rg/terragrunt.hcl, i got this is the error:

diego@AC-DMlagos2:/mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg$ terragrunt plan
[terragrunt] [/mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg] 2019/06/28 12:25:09 Running command: terraform --version
[terragrunt] 2019/06/28 12:25:12 Reading Terragrunt config file at /mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg/terragrunt.hcl
[terragrunt] 2019/06/28 12:25:12 Cleaning up existing *.tf files in /mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg/.terragrunt-cache/zAZxgP0FApqtssVNnTa-yrc98c4/RbFKWdQdmvbTH-w05FIEKUUoxKo
[terragrunt] 2019/06/28 12:25:12 Downloading Terraform configurations from file:///mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-modules into /mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg/.terragru
nt-cache/zAZxgP0FApqtssVNnTa-yrc98c4/RbFKWdQdmvbTH-w05FIEKUUoxKo
[terragrunt] 2019/06/28 12:25:12 Copying files from /mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg into /mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg/.terragrunt-cache/zAZxgP0FApqts
sVNnTa-yrc98c4/RbFKWdQdmvbTH-w05FIEKUUoxKo/resource_group
[terragrunt] 2019/06/28 12:25:12 Setting working directory to /mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg/.terragrunt-cache/zAZxgP0FApqtssVNnTa-yrc98c4/RbFKWdQdmvbTH-w05FIEKUUoxKo/resource_group
[terragrunt] [/mnt/c/code/public.cloud/terraform-lab/diego-azure-tf-live/westus2/rg] 2019/06/28 12:25:12 Running command: terraform init

Initializing the backend...
container_name
  The container name.

temporary solution

But if i put the configuration of root/terragrunt.hcl (so the backend configuration) into the file root/westus2/terragrunt.hcl, everithing works as expected. I think that terragrunt was not able to completely reconstruct the whole folder hierarchy.

[robmorgan]

@robmorgan Could this be a bug with the recently added GCP/GCS support?

I've got something strange happening at the moment with GCS config in the same folder and its due to some of the extended configuration not being passed in correctly, but I'm still debugging it.

[xyklex]

I've got something strange happening at the moment with GCS config in the same folder and its due to some of the extended configuration not being passed in correctly, but I'm still debugging it.

Thanks.

@brikis98 I set TF_LOG=DEBUG with my plan command and noted that remote state backend is passed as arguments to terraform init as:

-backend-config=prefix=terraform-state/sandbox/terraform-instance/terraform.tfstate", "-backend-config=bucket=devops-dialonce", "-backend-config=credentials=/Users/monkey/tf-test/live/../creds.json"

Was this always the behavior to remote state backend instead of mixing the content in .terragrunt-cache as I thought it should be?

[robmorgan]

I have a feeling it may be related to: https://github.com/gruntwork-io/terragrunt/commit/2dd80cf37de675266bb21c89319e4cddf4e18d0c, but so far I have not been able to reproduce it consistently.