gruntwork-io / terragrunt

Terragrunt is a flexible orchestration tool that allows Infrastructure as Code written in OpenTofu/Terraform to scale.
https://terragrunt.gruntwork.io/
MIT License
7.78k stars 959 forks source link

Terragrunt does not override dependency blocks following _envcommon pattern #2510

Open iangrunt opened 1 year ago

iangrunt commented 1 year ago

I have this block in my _envcommon for eks-core-services

dependency "aurora" {
  config_path = "${get_terragrunt_dir()}/../../data-stores/aurora"

  mock_outputs = {
    primary_endpoint = "database"
    port             = 5432
  }
  mock_outputs_allowed_terraform_commands = ["validate", ]
}

I need to override this in my eks-core-services/terragrunt.hcl to point to a different location in my Reference Architecture, config_path = "${get_terragrunt_dir()}/../../data-stores/aurora-legacy":

terraform {
  source = "${include.envcommon.locals.source_base_url}?ref=v0.91.0"
}

dependency "aurora" {
  config_path = "${get_terragrunt_dir()}/../../data-stores/aurora-legacy"

  mock_outputs = {
    primary_endpoint = "database"
    port             = 5432
  }
  mock_outputs_allowed_terraform_commands = ["validate", ]
}

# Include the root `terragrunt.hcl` configuration, which has settings common across all environments & components.
include "root" {
  path = find_in_parent_folders()
}

# Include the component configuration, which has settings that are common for the component across all environments
include "envcommon" {
  path = "${dirname(find_in_parent_folders())}/_envcommon/services/eks-core-services.hcl"
  # We want to reference the variables from the included config in this configuration, so we expose it.
  expose = true
}

However, this configuration does not work, and Terragrunt still searches for the original dependency.

Tomasz-Kluczkowski commented 8 months ago

hit the same issue, the only way to override the dependency path is to use merge strategy = "deep" but then I get a lot of stuff that I don't want to get ...

aryanchirania commented 2 months ago

Hi @Tomasz-Kluczkowski, I am facing similar issue when trying to override config_path in the dependency block and while troubleshooting I found your merge_strategy sugesstion, but even after using merge_strategy as deep I am not able to override the config_path. Is there any specific version of terragrunt from which the override functionality was enabled with the deep merge strategy?

It will be very helpful if you can suggest something as I have a lot of use cases for this and currently as a workaround, I hvae to replicate the whole parent file just because of dependencies.

Tomasz-Kluczkowski commented 2 months ago

@aryanchirania I can show code example from our terragrunt project which changes the config_path.

_envcommon:

terraform {
  source = "${local.base_source_url}?ref=${local.vars.tf_module_version}"
}

dependency "aks" {
  config_path  = "../../../aks"
  skip_outputs = true
}

locals {
  vars = merge(
    read_terragrunt_config(find_in_parent_folders("global.hcl")).locals,
    read_terragrunt_config(find_in_parent_folders("cluster.hcl")).locals,
    read_terragrunt_config(find_in_parent_folders("env.hcl")).locals,
  )
  base_source_url = "<REDACTED>"
}

inputs = {
  orchestrator_version = local.vars.orchestrator_version
}

overridden config_path in one of the non-standard use cases:

include "root" {
  path = find_in_parent_folders()
}

include "envcommon" {
  path           = "${dirname(find_in_parent_folders())}/_envcommon/auto_scaling_nodepools.hcl"
  merge_strategy = "deep"
}

dependency "aks" {
  config_path  = "../aks"
  skip_outputs = true
}

inputs = {
...
...
...

Hope this helps.

aryanchirania commented 2 months ago

@Tomasz-Kluczkowski Thanks for sharing the example. I figured out why I was getting the error even after using merge_strategy = "deep, I was using expose = true as well in the include block and it seems to be the case that if expose is set to true then terragrunt doesn't overrides the parent dependencies.

Have removed the expose pararmeter now and it is working as expected.

ervinb commented 5 days ago

Apparently the dependency block is getting a special treatment when using the deep merge strategy, where a dependency can be referenced in both directions: parent -> child, child -> parent.

I bet this is why expose isn't behaving as expected.

https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#limitations-on-accessing-exposed-config image