gruntwork-io / terragrunt

Terragrunt is a flexible orchestration tool that allows Infrastructure as Code written in OpenTofu/Terraform to scale.
https://terragrunt.gruntwork.io/
MIT License
8.04k stars 975 forks source link

Destroy command failing to find file when using dependency #2508

Closed jtyr closed 1 year ago

jtyr commented 1 year ago

I have the following setup:

├── terraform
│   ├── envs
│   │   └── dev
│   │       ├── base
│   │       │   └── main.tf
│   │       └── cluster
│   │           └── main.tf
│   └── states
│       ├── dev-base.state
│       └── dev-cluster.state
└── terragrunt
    ├── envs
    │   └── dev
    │       ├── base
    │       │   ├── terragrunt.hcl
    │       │   └── tier.hcl
    │       ├── cluster
    │       │   ├── terragrunt.hcl
    │       │   └── tier.hcl
    │       └── env.hcl
    ├── org.hcl
    └── terragrunt.hcl

Where the files content is as follows:

# terraform/envs/dev/base/main.tf      
output "some_output" {
  value = "something"
}
# terraform/envs/dev/cluster/main.tf 
variable "some_input" {}

output "some_output" {
  value = "${var.some_input} else"
}
# terragrunt/envs/dev/base/terragrunt.hcl    
terraform {
  source = "../../../../terraform//envs/dev/base"
}

include {
  path = find_in_parent_folders()
}
# terragrunt/envs/dev/cluster/terragrunt.hcl 
terraform {
  source = "../../../../terraform//envs/dev/cluster"
}

include {
  path = find_in_parent_folders()
}

dependency "base" {
  config_path = "../base"
}

inputs = {
  some_input = dependency.base.outputs.some_output
}
# terragrunt/terragrunt.hcl                  
locals {
  org_vars = read_terragrunt_config(find_in_parent_folders("org.hcl"))
  env_vars = read_terragrunt_config(find_in_parent_folders("env.hcl"))
  tier_vars = read_terragrunt_config("tier.hcl")

  organization_unit = local.org_vars.locals.organization_unit
  environment       = local.env_vars.locals.environment
  tier              = local.tier_vars.locals.tier
}

generate "provider" {
  path      = "terraform.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<-EOF
    terraform {
      backend "local" {
        path = "../../../../../../../../../../terraform/states/${local.environment}-${local.tier}.state"
      }
    }
    EOF
}

inputs = merge(
  local.org_vars.locals,
  local.env_vars.locals,
  local.tier_vars.locals,
)
# terragrunt/envs/dev/base/tier.hcl    
locals {
  tier = "base"
}
# terragrunt/envs/dev/cluster/tier.hcl       
locals {
  tier = "cluster"
}
# terragrunt/envs/dev/env.hcl       
locals {
  environment = "dev"
}
# terragrunt/org.hcl          
locals {
  organization_unit = "test"
}

When I run terragrunt apply from the terragrunt/envs/dev/base directory and then from the terragrunt/envs/dev/cluster directory, all works just fine. But once I run terragrunt destroy from the terragrunt/envs/dev/cluster directory, I get an error the it cannot find files used in the included terragrunt.hcl but the destruction works just fine:

ERRO[0000] Error: Error in function call
       prefix=[/tmp/test/terragrunt] 
ERRO[0000]   on /tmp/test/terragrunt/terragrunt.hcl line 2, in locals:  prefix=[/tmp/test/terragrunt] 
ERRO[0000]    2:   org_vars = read_terragrunt_config(find_in_parent_folders("org.hcl"))  prefix=[/tmp/test/terragrunt] 
ERRO[0000]                                               prefix=[/tmp/test/terragrunt] 
ERRO[0000] Call to function "find_in_parent_folders" failed: ParentFileNotFound: Could not find a org.hcl in any of the parent folders of /tmp/test/terragrunt/terragrunt.hcl. Cause: Traversed all the
way to the root..
  prefix=[/tmp/test/terragrunt] 
ERRO[0000] Encountered error while evaluating locals in file /tmp/test/terragrunt/terragrunt.hcl  prefix=[/tmp/test/terragrunt] 

Initializing the backend...

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Initializing the backend...

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Changes to Outputs:
  - some_output = "something else" -> null

You can apply this plan to save these new output values to the Terraform
state, without changing any real infrastructure.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value:

But if I comment out the dependency related blocks from the terragrunt/envs/dev/cluster/terragrunt.hcl file:

terraform {
  source = "../../../../terraform//envs/dev/cluster"
}

include {
  path = find_in_parent_folders()
}

#dependency "base" {
#  config_path = "../base"
#}

inputs = {
  #some_input = dependency.base.outputs.some_output
  some_input = "something"
}

the terragrunt destroy command is showing no error:


Initializing the backend...

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Changes to Outputs:
  - some_output = "something else" -> null

You can apply this plan to save these new output values to the Terraform
state, without changing any real infrastructure.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: 

It looks like when the dependency is used, it somehow evaluates the destroy command from the perspective of the included terragrunt.hcl file instead of from the perspective of the root terragrunt.hcl file.

I tried to use the Terragrunt built-in functions (e.g. get_terragrunt_dir, get_parent_terragrunt_dir, get_original_terragrunt_dir, path_relative_to_include, path_relative_from_include) in the paths specified in the terraform and dependency blocks but nothing really helped.

This issue is similar to #2330 and #1891 with the difference that I don't specify the --terragrunt-working-dir at all as I run the commands directly from the root module.

macropain commented 1 year ago

I'm seeing exact same behaviour.


terragrunt --version
terragrunt version v0.45.0

terraform --version
Terraform v1.3.0
on darwin_arm64```
levkohimins commented 1 year ago

Hi @jtyr

Thank you for the detailed explanation, it was really helpful. Issue resolved in v0.45.17 release.

You also need to update the config file.

# terragrunt/terragrunt.hcl        
locals {
  org_vars = read_terragrunt_config("${get_parent_terragrunt_dir()}/org.hcl")
  env_vars = read_terragrunt_config("${get_parent_terragrunt_dir()}/envs/dev/env.hcl")
  tier_vars = read_terragrunt_config("tier.hcl")
  ...
}
...

Good luck!