Terraform extra_arguments ENV Variables lost if dependency block is added

[pipiobjo]

Describe the bug Auth Context is not propaged properly after adding dependency to terragrunt module.

To Reproduce

Sample setup to reproduce the issue, which simple creates two resource groups in Azure. Which should use the same Service Principal for authentication. Separated in two terragrunt modules.

Given following file structure:

├── 01-base
│   └── terragrunt.hcl
├── 02-network
│   └── terragrunt.hcl
├── backend-storage.hcl
├── dev.hcl
└── sp.hcl

01-base/terragrunt.hcl

include "dev" {
  path   = "${get_repo_root()}/iac/env/dev/dev.hcl"
  expose = true
}

locals {
  backend_vars = read_terragrunt_config("${get_repo_root()}/iac/env/dev/backend-storage.hcl")
  global = read_terragrunt_config(find_in_parent_folders("global.hcl"))
  sp_vars      = read_terragrunt_config(find_in_parent_folders("sp.hcl"))
  stageId      = "dev"
}

terraform {
  source = "${local.global.inputs.modules_source_base_url}base${local.global.inputs.modules_source_url_appendix}"
}

inputs = {
  company_code                 = local.global.inputs.company_code
  env                          = "dev"
  additional_information       = ""
  location_code                = local.global.inputs.location_code
  location                     = local.global.inputs.location
  tenant_id                    = local.global.inputs.tenant_id
  #  tenant_id_test                    = dependency.shared.outputs.tenant_id
  sp_object_id                 = local.sp_vars.inputs.SP_DEV_OBJ_ID
  vnet_address_space           = local.global.inputs.stages.dev.vnet_address_space
  bastion_subnet_address_space = local.global.inputs.stages.dev.bastion_subnet_address_space
  vm_subnet_address_space      = local.global.inputs.stages.dev.vm_subnet_address_space
  operating_team_group_id      = local.global.inputs.operating_team.object_id
  bastion_vm_size              = local.global.inputs.stages.dev.bastion_vm_size
  sp_k8s_dev_id                = local.global.inputs.stages.dev.k8s.service_principal_id
}

02-network/terragrunt.hcl

include "dev" {
  path   = "${get_repo_root()}/iac/env/dev/dev.hcl"
  expose = true
}

locals {
  global       = read_terragrunt_config(find_in_parent_folders("global.hcl"))
  backend_vars = read_terragrunt_config("${get_terragrunt_dir()}/../backend-storage.hcl")
  sp_vars      = read_terragrunt_config(find_in_parent_folders("sp.hcl"))
  stageId      = "dev"
}

terraform {
  source = "${local.global.inputs.modules_source_base_url}network${local.global.inputs.modules_source_url_appendix}"
}

inputs = {
  rg_name     = "test-rg"
  rg_location = "westeurope"
}

dev.hcl

locals {
  backend_vars = read_terragrunt_config("${get_repo_root()}/iac/env/dev/backend-storage.hcl")
  global = read_terragrunt_config(find_in_parent_folders("global.hcl"))
  sp_vars      = read_terragrunt_config("${get_repo_root()}/iac/env/dev/sp.hcl")
  stageId      = "dev"
}

#
remote_state {
  backend = "azurerm"

  generate = {
    path      = "${get_terragrunt_dir()}/backend.tf"
    if_exists = "overwrite"
  }
  config = {
    use_azuread_auth     = true
    subscription_id      = local.global.inputs.stages.dev.subscription_id
    resource_group_name  = local.backend_vars.inputs.rg_name        # "rg-terraform-dev"
    storage_account_name = local.backend_vars.inputs.sa_name        # "satfazurecaftexas"
    container_name       = local.backend_vars.inputs.container_name #"tf-state"
    key                  = "${replace(path_relative_to_include(), "/[^0-9A-Za-z]/", "")}.tfstate"
  }
}

terraform {
  extra_arguments "env_vars" {
    commands = [
      "init",
      "apply",
      "refresh",
      "import",
      "plan",
      "taint",
      "untaint",
      "destroy"
    ]

    env_vars = {
      ARM_CLIENT_ID       = local.sp_vars.inputs.SP_DEV_ID
      ARM_CLIENT_SECRET   = local.sp_vars.inputs.SP_DEV_SECRET
      ARM_TENANT_ID       = local.global.inputs.tenant_id
      ARM_SUBSCRIPTION_ID = local.global.inputs.stages.dev.subscription_id
      ARM_USE_CLI = true
      TF_LOG = "trace"
    }
  }
}

tf-modules/base/main.tf

terraform {
  required_version = ">= 1.4"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.59.0"
    }
  }
}

provider "azurerm" {
  features {}
}

locals {
  additional_information = var.additional_information == "" ? "" : "-${var.additional_information}"
}

resource "azurerm_resource_group" "ti" {
  name     = "rg-${lower(var.company_code)}-${lower(var.env)}-${lower(var.location_code)}${local.additional_information}"
  location = var.location # "West Europe"
}

output "rg_name" {
  value = azurerm_resource_group.ti.name
}

output "rg_location" {
  value = azurerm_resource_group.ti.location
}

variable "company_code" {
  type        = string
  description = "Code of the company"
  default     = ""
}

variable "env" {
  type        = string
  description = "Name of environment"
  default     = ""
}

variable "location_code" {
  type        = string
  description = "Code of the location"
  default     = ""
}

variable "location" {
  type        = string
  description = "location"
  default     = ""
}

variable "additional_information" {
  type        = string
  description = "Name of business area"
  default     = ""
}

tf-modules/network/main.tf

terraform {
  required_version = ">= 1.4"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.59.0"
    }
  }
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "ti" {
  name     = var.rg_name
  location = var.rg_location
}

output "rg_name_test" {
  value = var.rg_name
}

variable "rg_name" {
  type        = string
  description = "Name of resource group"
  default     = ""
}

variable "rg_location" {
  type        = string
  description = "Location of resource group"
  default     = ""
}

Execute apply with following commands in every subfolder:

  echo "${BLUE}APPLY: Running terragrunt init - $MODULE_PATH ${NO_COLOR}"
  TERRAGRUNT_TFPATH="${TERRAFORM_EXECUTABLE}" "${TERRAGRUNT_EXECUTABLE}" init \
    --terragrunt-working-dir $IAC_ENV_MODULE_DIR \
    --terragrunt-include-external-dependencies \
    --terragrunt-non-interactive \
    --terragrunt-log-level=debug

  echo "${BLUE}APPLY: Running terragrunt refresh - $MODULE_PATH ${NO_COLOR}"
  TERRAGRUNT_TFPATH="${TERRAFORM_EXECUTABLE}" "${TERRAGRUNT_EXECUTABLE}" refresh \
      --terragrunt-working-dir $IAC_ENV_MODULE_DIR \
      --terragrunt-include-external-dependencies \
      --terragrunt-non-interactive \
      --terragrunt-log-level=debug

  echo "${BLUE}APPLY: Running terragrunt apply- $MODULE_PATH ${NO_COLOR}"
  TERRAGRUNT_TFPATH="${TERRAFORM_EXECUTABLE}" "${TERRAGRUNT_EXECUTABLE}" apply \
    -auto-approve \
    -input=false \
    --terragrunt-working-dir $IAC_ENV_MODULE_DIR \
    --terragrunt-include-external-dependencies \
    --terragrunt-non-interactive \
    --terragrunt-log-level=trace \
    --terragrunt-debug

Leads to both modules being applied and both resource groups being created. Everything fine until here.

Now I want to use the output of the base module in the network module. For that I add the following to the network module:

02-network/terragrunt.hcl

dependency "base" {
  config_path = "${get_repo_root()}/iac/env/dev/01-base"
  mock_outputs = {
    rg_name = "fake-vpc-id"
    rg_location = "fake-vpc-id"
  }
}

the complete file looks like this:

include "dev" {
  path   = "${get_repo_root()}/iac/env/dev/dev.hcl"
  expose = true
}

dependency "base" {
  config_path = "${get_repo_root()}/iac/env/dev/01-base"
  mock_outputs = {
    rg_name = "fake-vpc-id"
    rg_location = "fake-vpc-id"
  }
}

locals {
  global       = read_terragrunt_config(find_in_parent_folders("global.hcl"))
  backend_vars = read_terragrunt_config("${get_terragrunt_dir()}/../backend-storage.hcl")
  sp_vars      = read_terragrunt_config(find_in_parent_folders("sp.hcl"))
  stageId      = "dev"
}

terraform {
  source = "${local.global.inputs.modules_source_base_url}network${local.global.inputs.modules_source_url_appendix}"

}

inputs = {
  rg_name     = "test-rg"
  rg_location = "westeurope"

}

The first modules finished successfully with

Your infrastructure matches the configuration

Apply the code with the same commands as before, leads to following error:

Error: Error building ARM Config: obtain subscription(c4c0eeb3-28ed-49f4-b93f-196f46578cd2) from Azure CLI: parsing json result from the Azure CLI: waiting for the Azure CLI: exit status 1: ERROR: Please run 'az login' to setup account.

Following file structure is created:

2023-06-07T13:45:31.1945800Z ├── 01-base
2023-06-07T13:45:31.1945989Z │   ├── .terraform.lock.hcl
2023-06-07T13:45:31.1946191Z │   ├── .terragrunt-cache
2023-06-07T13:45:31.1946416Z │   │   └── ANhtz_Bs0oBfyvLKnQ1x7AykYko
2023-06-07T13:45:31.1946657Z │   │       └── pCGfaF90pqMnYKubzGd86NIWwXc
2023-06-07T13:45:31.1946882Z │   │           ├── .terraform
2023-06-07T13:45:31.1947072Z │   │           │   ├── providers
2023-06-07T13:45:31.1947300Z │   │           │   │   └── registry.terraform.io
2023-06-07T13:45:31.1947720Z │   │           │   │       └── hashicorp
2023-06-07T13:45:31.1947936Z │   │           │   │           └── azurerm
2023-06-07T13:45:31.1948153Z │   │           │   │               └── 3.59.0
2023-06-07T13:45:31.1948384Z │   │           │   │                   └── linux_amd64
2023-06-07T13:45:31.1948655Z │   │           │   │                       └── terraform-provider-azurerm_v3.59.0_x5
2023-06-07T13:45:31.1948896Z │   │           │   └── terraform.tfstate
2023-06-07T13:45:31.1949114Z │   │           ├── .terraform.lock.hcl
2023-06-07T13:45:31.1949345Z │   │           ├── .terragrunt-module-manifest
2023-06-07T13:45:31.1949583Z │   │           ├── .terragrunt-source-manifest
2023-06-07T13:45:31.1949818Z │   │           ├── .terragrunt-source-version
2023-06-07T13:45:31.1950041Z │   │           ├── backend.tf
2023-06-07T13:45:31.1950232Z │   │           ├── main.tf
2023-06-07T13:45:31.1950431Z │   │           ├── output.tf
2023-06-07T13:45:31.1950632Z │   │           ├── terragrunt.hcl
2023-06-07T13:45:31.1950840Z │   │           ├── tfplan
2023-06-07T13:45:31.1951035Z │   │           ├── tfplan.out
2023-06-07T13:45:31.1951231Z │   │           └── variables.tf
2023-06-07T13:45:31.1951419Z │   ├── backend.tf
2023-06-07T13:45:31.1951608Z │   ├── terragrunt.hcl
2023-06-07T13:45:31.1951799Z │   └── tfplan.out
2023-06-07T13:45:31.1951980Z ├── 02-network
2023-06-07T13:45:31.1952170Z │   └── terragrunt.hcl
2023-06-07T13:45:31.1952360Z ├── backend-storage.hcl
2023-06-07T13:45:31.1952538Z ├── dev.hcl
2023-06-07T13:45:31.1952708Z └── sp.hcl

If I deactivate the terraform extra_arguments block and set the ENV Variables directly. Everything works as expected. But of course a dynamic switch of different Auth Contexts between modules is not possible.

Expected behavior terraform extra_arguments are evaluated per occurrence, referenced by include block or assigned directly

Versions

• Terragrunt version: 0.46.2
• Terraform version: 1.4.6
• Environment details: Ubuntu, Azure DevOps Pipeline

[EdanBrooke]

Hello,

I am also experiencing this behaviour when using Terragrunt extra_arguments to configure remote state authentication information for GitLab's HTTP-based backend using existing GITLAB_USERNAME and GITLAB_ACCESS_TOKEN environment variables.

terraform {
  extra_arguments "set_env" {
    commands = ["init", "plan", "apply", "refresh", "import", "destroy", "show", "state", "taint", "untaint"]
    env_vars = {
      # GITLAB_USERNAME should be passed as TF_HTTP_USERNAME
      TF_HTTP_USERNAME = get_env("GITLAB_USERNAME")
      # GITLAB_PASSWORD should be passed as TF_HTTP_PASSWORD
      TF_HTTP_PASSWORD = get_env("GITLAB_ACCESS_TOKEN")
    }
  }
}

When I include the following dependency, remote authentication fails for the core module's state backend.

dependency "core" {
  config_path = "../core"
}

For now, I am working around the issue by manually setting TF_HTTP_USERNAME and TF_HTTP_PASSWORD. Is this a bug or is this intended behaviour?

Best regards, Edan

[nick4fake]

Seems like a very serious bug, as there is no way to use dependencies with extra_arguments

[nick4fake]

Related: https://github.com/gruntwork-io/terragrunt/issues/1932

[nick4fake]

In my case this worked: https://github.com/gruntwork-io/terragrunt/issues/1590#issuecomment-800856330

disable_dependency_optimization = true

But this is obviously not a good solution?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for raising this issue.

[EdanBrooke]

I believe this is something we would still like to see? I will aim to re-test.