Initialization required when TF_DATA_DIR is set

Background:

We have been having issues with terragrunt performance for a while, and we have downgraded from v0.26 to v0.23.34. I noticed newer versions keep downloading providers, even when they are already init'ed. I believe it's related to #1322, which was fixed at v0.28.3 When creating a small example to post here, I'm using the newest version of terragrunt (v0.28.12), and I believe I isolated the problem to TF_DATA_DIR. I also found #1577, but there is not enough information there, and thus this issue may be related.

Issue

Terragrunt does not evaluate dependencies correctly, and throws Error: Initialization required. Please see the error message above. Note: this uses azurerm as backend. As long as it's a remote backend, you will be able to reproduce the issue. It seems not to happen on local backends.

Reproducing

Basically 3 sections, where app2 depends on app1 and both apps depends on infra. The main.tf are basically a minimum config for terraform and variables and outputs to create a proper dependency

Folder Structure

├── apps
│   ├── app1
│   │   ├── main.tf
│   │   └── terragrunt.hcl
│   └── app2
│       ├── main.tf
│       └── terragrunt.hcl
├── infra
│   ├── main.tf
│   └── terragrunt.hcl
└── terragrunt.hcl

apps/app1/main.tf

terraform {
  backend "azurerm" {}

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.44.0"
    }
  }
  required_version = ">= 0.13"
}

variable "infra_value" {
  description = "the infra value"
  type        = string
}

output "app_value" {
  value = "${var.infra_value}"
}

apps/app1/terragrunt.hcl

include {
  path = find_in_parent_folders()
}
dependency "infra" {
  config_path = "${get_parent_terragrunt_dir()}/infra"
}
inputs = {
  infra_value = dependency.infra.outputs.infra_value
}

apps/app2/main.tf

terraform {
  backend "azurerm" {}

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.44.0"
    }
  }
  required_version = ">= 0.13"
}

variable "infra_value" {
  description = "the infra value"
  type        = string
}

variable "app1_value" {
  description = "the app1 value"
  type        = string
}

output "app_value" {
  value = "${var.infra_value}-${var.app1_value}"
}

apps/app2/terragrunt.hcl

include {
  path = find_in_parent_folders()
}
dependency "infra" {
  config_path = "${get_parent_terragrunt_dir()}/infra"
}
dependency "app1" {
  config_path = "${get_terragrunt_dir()}/../app1/"
}
inputs = {
  infra_value = dependency.infra.outputs.infra_value
  app1_value  = dependency.app1.outputs.app_value
}

infra/main.tf

terraform {
  backend "azurerm" {}

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.44.0"
    }
  }
  required_version = ">= 0.13"
}

variable "environment" {
  description = "the environment"
  type        = string
}

variable "region" {
  description = "the region"
  type        = string
}

output "infra_value" {
  value = "${var.region}-${var.environment}"
}

infra/terragrunt.hcl

include {
  path = find_in_parent_folders()
}

terragrunt.hcl

remote_state {
  backend = "azurerm"
  config = {
    resource_group_name  = "<RG here>"
    storage_account_name = "<StorageAccount here>"
    container_name       = "test"
    key                  = lower("${get_env("TF_VAR_region")}/${get_env("TF_VAR_environment")}/${path_relative_to_include()}.tfstate")
    subscription_id      = "<subscriptionID here>"
    tenant_id            = "<tenantID here>"
  }
}

terraform {
  extra_arguments "extra_env_vars" {
    # Below is a list of all terraform commands excluding "version" and "push"
    commands = ["apply", "console", "destroy", "env", "fmt", "get", "graph", "import", "init",
      "login", "logout", "output", "plan", "providers", "refresh", "show", "taint",
      "untaint", "validate", "workspace", "debug", "force-unlock", "state"
    ]
    env_vars = {
      TF_DATA_DIR = ".terraform/${get_env("TF_VAR_region")}/${get_env("TF_VAR_environment")}"
    }
  }
}

Executing

At the root module, set the environments TF_VAR_region and TF_VAR_environment to anything (i.e export TF_VAR_region=us TF_VAR_environment=dev) and execute: terrragrunt run-all apply --terragrunt-non-interactive you get the error: Error: Initialization required. Please see the error message above.

Making it work

Remove any .terragrunt-cache and .terraform folders from you local

find . -name .terraform | xargs rm -rf
find . -name .terragrunt-cache | xargs rm -rf

Comment out the TF_DAT_DIR from the root terragrunt.hcl and execute terrragrunt run-all apply --terragrunt-non-interactive again.

Alternativelly, using the version we are currently using (terragrunt v0.23.34) execute terrragrunt apply-all --terragrunt-non-interactive This version of terragrunt works fine with TF_DATA_DIR

gruntwork-io / terragrunt