remote_state path isn't being recalculated correctly when a remote_state block exists in terragrunt cache

I have a setup where I'm using input variables to set the AWS region and account_id to avoid having to duplicate terragrunt files. this caused an issue when i upgraded to a terragrunt version that includes the new dependency output retrieval optimization - say i have module A that outputs something like a VPC ID, and module B that consumes that and uses it to create a subnet. if I apply module A in region us-east-1 and AWS account 123, then apply it again in us-west-2 and AWS account 456, then I go to module B and apply it in us-east-1 and AWS account 123, I should expect for the subnet created in module B to be created in the us-east-1 subnet in account 123 - instead it shows up in us-west-2 and account 456 (or in my case I get an error having to do with cross-account permissions but still, that's what it tried to do).

the reason for this seems to be that the remote_state block created in the terragrunt cache as part of the second terragrunt apply of module A is being used as part of the state path calculation for getting outputs to use in module B as input variables. I'm confident this is true because if i go into the terragrunt cache and manually edit the remote_state block to match the region and account_id I'm applying module B in, then everything works correctly.

as a workaround I've taken to setting disable_dependency_optimization to true, but regardless I think this issue is still an indication of an unintended behavior; also, the dependency output optimization is great, and I'd love to be able to use it. it makes the terragrunt run output much cleaner because there isn't a bunch of terraform init + output spam from dependent modules, plus it's much faster.

my setup has a lot of stuff not relevant to this issue that I'd rather not share, but I've created a pared down example that should be the minimum required setup to reproduce this issue. the directory structure looks like this:

.
├── deployment
│   ├── subnet
│   │   └── terragrunt.hcl
│   └── vpc
│       └── terragrunt.hcl
├── modules
│   ├── subnet
│   │   └── subnet.tf
│   └── vpc
│       └── vpc.tf
└── terragrunt.hcl

the parent terragrunt.hcl file contains:

locals {
  region     = get_env("REGION", "MISSING")
  account    = get_env("ACCOUNT", "MISSING")
  account_id = get_env("ACCOUNT_ID", "MISSING")
}

remote_state {
  backend                         = "s3"
  disable_dependency_optimization = false

  generate = {
    path      = "remote_state.tf"
    if_exists = "overwrite"
  }

  config = {
    bucket         = "blah-tf"
    key            = "remote_state_test/${local.account}/${local.region}/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    dynamodb_table = "terraform-lock"
  }
}

generate "provider" {
  path      = "provider.tf"
  if_exists = "skip"
  contents  = <<EOF
provider "aws" {
  region              = "${local.region}"
  allowed_account_ids = [${local.account_id}]

  assume_role {
    role_arn     = "arn:aws:iam::${local.account_id}:role/OrganizationAccountAccessRole"
    session_name = "terraform"
  }
}
EOF
}

generate "provider_versions" {
  path      = "provider_versions.tf"
  if_exists = "skip"
  contents  = <<EOF
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "= 3.70.0"
    }
  }

  required_version = "= 1.1.0"
}
EOF
}

deployment/subnet/terragrunt.hcl:

include "global_configs" {
  path           = find_in_parent_folders("terragrunt.hcl")
  merge_strategy = "deep"
  expose         = true
}

terraform {
  source = "${find_in_parent_folders("modules")}//subnet"
}

dependencies {
  paths = [
    "${find_in_parent_folders("deployment")}/vpc",
  ]
}

dependency "vpc" {
  config_path = "${find_in_parent_folders("deployment/")}/vpc"
}

inputs = {
  vpc_id = dependency.vpc.outputs.vpc_id
}

deployment/vpc/terragrunt.hcl:

include "global_configs" {
  path           = find_in_parent_folders("terragrunt.hcl")
  merge_strategy = "deep"
  expose         = true
}

terraform {
  source = "${find_in_parent_folders("modules")}//vpc"
}

modules/subnet/subnet.tf:

variable "vpc_id" {}

resource "aws_subnet" "main" {
  vpc_id     = var.vpc_id
  cidr_block = "10.0.1.0/24"

  tags = {
    Name = "Main"
  }
}

modules/vpc/vpc.tf:

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

output "vpc_id" {
  value = aws_vpc.main.id
}

to reproduce the state problem, run (replacing account ids with real ids ofc):

cd deployment/vpc
REGION=us-east-1 ACCOUNT=account-1 ACCOUNT_ID=123 terragrunt apply
REGION=us-west-2 ACCOUNT=account-2 ACCOUNT_ID=456 terragrunt apply
cd ../subnet
REGION=us-east-1 ACCOUNT=account-1 ACCOUNT_ID=123 terragrunt apply --terragrunt-source-update

the last command should either produce some kind of error, or create resources in the wrong region/account. then cat out the remote_state.tf file in ../vpc/[whatever the cache path is], see that it's in us-west-2 still, and try editing it to the right region and account id, and reapplying subnet - it should work as intended

gruntwork-io / terragrunt

remote_state path isn't being recalculated correctly when a remote_state block exists in terragrunt cache #2022