search

antonbabenko / terragrunt-reference-architecture

Terragrunt Reference Architecture (upd: May 2020)
MIT License
375 stars 104 forks source link

Problem loading configurations from a parent directory #8

Closed sheldonhull closed 3 years ago

sheldonhull commented 3 years ago

This file uses commands to load common.tfvars and regional.tfvars

However, it seems to have a problem loading this common.tfvars in when it runs, saying it can't find common.tfvars. 

Call to function "find_in_parent_folders" failed

I just tried navigating into the region level directory under the "parent" terragrunt.hcl file. It errored out and no detail on error, just this

time=2021-04-16T12:39:14-05:00 level=error msg=Encountered the following errors:
Hit multiple errors:
Hit multiple errors:
exit status 127

It also generated a backend file in the "parent" directory that has the common variables, but is not the individual region's section.

When I copied the debug command output and run the command below it plans fine. 

terraform plan -var-file="/Users/sheldonhull/git/repo/terragrunt/staging/eu-central-1/vpc/terragrunt-debug.tfvars.json" "/Users/sheldonhull/git/repo/terragrunt/staging/eu-central-1/vpc/.terragrunt-cache/QaAhZIo4_imktYigJ0DeKyQKkDk/ky_VpYadOztK_rvSkFtkAnFs6aY/modules/vpc"

This is a bit confusing as new terragrunt user. I'm not seeing the native terraform commands exposed easily to know what happened, it's just outputting the terraform help content.

The error doesn't even show the actual terraform init/plan command, just this:


DEBU[0001] Run this command to replicate how terraform was invoked:  prefix=[/Users/sheldonhull/git/repo/terragrunt/staging/eu-central-1/vpc]
DEBU[0001]  terraform  -var-file="/Users/sheldonhull/git/git/repo/terragrunt/staging/eu-central-1/vpc/terragrunt-debug.tfvars.json" "/Users/sheldonhull/git/terragrunt/staging/eu-central-1/vpc/.terragrunt-cache/QaAhZIo4_imktYigJ0DeKyQKkDk/ky_VpYadOztK_rvSkFtkAnFs6aY/modules/vpc"  prefix=[/Users/sheldonhull/git/repo/terragrunt/staging/eu-central-1/vpc]
panic: runtime error: index out of range [0] with length 0

goroutine 19 [running]:
github.com/gruntwork-io/terragrunt/options.(*TerragruntOptions).InsertTerraformCliArgs(0xc000365dc0, 0x2848b78, 0x0, 0x0)
    github.com/gruntwork-io/terragrunt/options/options.go:296 +0x37c
github.com/gruntwork-io/terragrunt/cli.runTerragruntWithConfig(0xc0003b4700, 0xc000365dc0, 0xc000894000, 0x0, 0xf, 0x0)
    github.com/gruntwork-io/terragrunt/cli/cli_app.go:620 +0xae
github.com/gruntwork-io/terragrunt/cli.RunTerragrunt(0xc0003b4700, 0x5, 0x1f21e28)
    github.com/gruntwork-io/terragrunt/cli/cli_app.go:474 +0x85c
github.com/gruntwork-io/terragrunt/configstack.(*runningModule).runNow(0xc0002e8820, 0xc000149f5f, 0x0)
    github.com/gruntwork-io/terragrunt/configstack/running_module.go:243 +0x185
github.com/gruntwork-io/terragrunt/configstack.(*runningModule).runModuleWhenReady(0xc0002e8820, 0xc0000a83c0)
    github.com/gruntwork-io/terragrunt/configstack/running_module.go:206 +0xd5
github.com/gruntwork-io/terragrunt/configstack.runModules.func1(0xc00088a410, 0xc0000a83c0, 0xc0002e8820)
    github.com/gruntwork-io/terragrunt/configstack/running_module.go:172 +0x5d
created by github.com/gruntwork-io/terragrunt/configstack.runModules
    github.com/gruntwork-io/terragrunt/configstack/running_module.go:170 +0x11b

Directory Structure

.                                                                                                   
├── README.md                                                                                       
├── Taskfile.yml                                                                                    
├── modules
│   ├── aws-data
│   │   ├── README.md
│   │   ├── main.tf
│   │   ├── outputs.tf
│   │   └── versions.tf
│   └── vpc
│       ├── README.md
│       ├── label.tf
│       ├── outputs.subnets.tf
│       ├── outputs.vpc.tf
│       ├── variables.common.tf
│       ├── variables.stack.tf
│       ├── versions.tf
│       └── vpc.tf
├── terragrunt
│   └── staging     <<<<<<<<<<< THIS IS MY WORKING DIRECTORY, running terragrunt from this context
│       ├── Taskfile.yml
│       ├── common.tfvars  <---- NEVER SEES TO FIND THIS
│       ├── eu-central-1   <---- navigating into this subdirectory and running doesn't change it's ability to find the common.tfvars
│       │   ├── regional.tfvars
│       │   └── vpc
│       │       ├── terragrunt-debug.tfvars.json
│       │       └── terragrunt.hcl
│       ├── global  <---- not doing anything with it, they are all commented out for now
│       │   ├── iam-account
│       │   │   └── terragrunt.hcl
│       │   ├── iam-assumable-roles
│       │   │   └── terragrunt.hcl
│       │   └── regional.tfvars
│       └── terragrunt.hcl  <---- what i'm calling "ROOT" so I can "run-all" against all the pieces in this stack
antonbabenko commented 3 years ago

Hi there!

You need to start debugging from an internal directory and try to get it to work first:

cd terragrunt/staging/eu-central-1/vpc
terragrunt init
terragrunt apply

Then you can go one level up and try to run terragrunt apply-all.

I don't think I have simple+working Terragrunt code in the open-source where common.tfvars is used but maybe you can see how I write it in other places, too: https://github.com/antonbabenko/modules.tf-demo https://github.com/terraform-aws-modules/meta

In general, the questions related to the terragrunt are better to ask on the official repository as there are more users who may help you.

sheldonhull commented 3 years ago

For future reference I realized I misread the commands.

I needed terragrunt run-all plan for this to work correctly. Now my output is correct. I didn't realize I was nesting 2 layers of subcommands.

The other bit someone in sweetops slack community mentioned was deprecation of using tfvars. Instead they said to use: 

inputs = merge(
  # Configure Terragrunt to use common vars encoded as yaml to help you keep often-repeated variables (e.g., account ID)
  # DRY. We use yamldecode to merge the maps into the inputs, as opposed to using varfiles due to a restriction in
  # Terraform >=0.12 that all vars must be defined as variable blocks in modules. Terragrunt inputs are not affected by
  # this restriction.
  yamldecode(
    file("${find_in_parent_folders("region.yaml", "empty.yaml")}"),
  ),
  yamldecode(
    file("${find_in_parent_folders("env.yaml", "empty.yaml")}"),
  ),

due to the changes in the behavior that block the tfvars from being pulled in correctly, while yaml works fine in loading into TF inputs.

I'm still having a problem with it loading a parent file from a child, but will work from your example and see if I can get this to work then.

sheldonhull commented 3 years ago

It worked this time from parent directory! Woot Woot. :tada:

Thanks again for the help.

antonbabenko commented 3 years ago

Good that it works!

Yes, yamldecode is an option if you want to deal with YAML or have a lot of variables.

I usually prefer to deal with hcl files using read_terragrunt_config() as I have here - https://github.com/terraform-aws-modules/meta/blob/master/github/repositories/terraform-aws-modules/terraform-aws-appsync/terragrunt.hcl#L10

sheldonhull commented 3 years ago

Much better. Prefer to keep it terraform focused and not yaml. Will give that a try

sheldonhull commented 3 years ago

That worked great and was much more intuitive as well. Thanks again as that helped unblocked me!

antonbabenko commented 3 years ago

You are welcome.

Maybe I should have one more episode about Terragrunt during Your Weekly Dose of Terraform. Who knows :)