gruntwork-io / terragrunt

Terragrunt is a flexible orchestration tool that allows Infrastructure as Code written in OpenTofu/Terraform to scale.
https://terragrunt.gruntwork.io/
MIT License
7.8k stars 962 forks source link

Only one level of includes is allowed #1366

Open nedaair opened 3 years ago

nedaair commented 3 years ago

1. This is my project structure.

terragrunt-project 
  - enviroment
     - ec2
        - terragrunt.hcl
     - s3
        - terragrunt.hcl
     - terragrunt.hcl
  - module
     - ec2
        - main.tf
        - outputs.tf
        - variables.tf

2. This is root terragrunt.hcl

generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "aws" {
  region = "ap-northeast-2"
}
EOF
}

remote_state {
  backend = "s3"
  config = {
    encrypt        = true
    bucket         = "terragrunt-test"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = "ap-northeast-2"
    dynamodb_table = "terragrunt-locks"
  }
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }
}

3. This is terragrunt.hcl in ec2 folder

terraform {
  source = "../../module/ec2"
}

inputs = {
  subnet_id = "subnet-2d11c946"
}

# Include all settings from the root terragrunt.hcl file
include {
  path = find_in_parent_folders()
}

I called "terragrunt apply-all" under the environment folder. Initially, it worked well and created an EC2 instance. However, the second execution results in the error as follows.

Error processing module at 'C:/Users/abc/PycharmProjects/terragrunt-study/enviroment/ec2/.terragrunt-cache/atTOD5LBQHAAgxKMQddh6R4j
XUE/mW46bbJv_XuxaHj-jku8hlgoUlQ/terragrunt.hcl'. How this module was found: Terragrunt config file found in a subdirectory of C:/Users/abc/PycharmProjects/terragrun
t-study/enviroment. Underlying error: C:/Users/abc/PycharmProjects/terragrunt-study/enviroment/ec2/.terragrunt-cache/atTOD5LBQHAAgxKMQddh6R4jXUE/mW46bbJv_XuxaHj-jku
8hlgoUlQ/terragrunt.hcl includes C:/Users/abc/PycharmProjects/terragrunt-study/enviroment/ec2/terragrunt.hcl, which itself includes C:/Users/abc/PycharmProj
ects/terragrunt-study/enviroment/ec2/terragrunt.hcl. Only one level of includes is allowed.

What's interesting is that Linux or Mac doesn't have a problem, but it only happens with Windows 10. My terragrunt version is v0.24.3 and terraform version is v0.13.2

brikis98 commented 3 years ago

How weird. Must be some sort of path issue on Windows. We don't have CI set up for Windows yet 😕

If someone has time to dig into this bug, a PR with a fix is very welcome!

vjaykumar-v commented 3 years ago

I am facing the same problem when plan was executed on independently (instance leve) then next time plan-all executed from higher level (region)

tomcart90 commented 3 years ago

I also hit this issue today on Windows. An initial plan-all worked, but then once the terragrunt cache had been populated further plan-all actions failed with the same error that @nedaair reports.

xmclark commented 3 years ago

Same issue, on windows 10. First plan-all or apply-all succeeds, but subsequent ones fail with @nedaair 's message. It seems to be resolving my generated terraform.hcl in the cache directories, and getting mad about it.

xmclark commented 3 years ago

@brikis98 I've started digging, and it seems that the config module tests don't succeed without signfiicant update. Having CircleCI run windows builds would ensure these tests pass. Your CI tool supports Windows builds...is that an option?

brikis98 commented 3 years ago

Yes, CircleCi does now support Windows images, so I suspect it's possible. However, I'm not sure when our team will have time to get to it and work out all the kinks with making the CI build pass. If anyone is interested, PRs are very welcome! https://circleci.com/docs/2.0/hello-world-windows/

phr34ker commented 3 years ago

Also facing the same issue on Win10

brandon-langley-mf commented 3 years ago

I was able to test and confirm that this issue at least appears to be caused because the .terragrunt-cache folders are being included and processed by *-all on subsequent runs. I've been able to work around it by adding this to my command line:

--terragrunt-download-dir c:\terragrunt-cache

It would be highly valuable to get this fixed, because it's far too easy to trip over on Windows and there's no intuitive way to figure out WHY it's giving you a hard time. Worse when your colleague is running on Linux having no issues with the same project :)

brandon-langley-mf commented 3 years ago

This seems to be solved by https://github.com/gruntwork-io/terragrunt/releases/tag/v0.26.4.

xmclark commented 3 years ago

I tried 0.26.4 windows_amd64 build and it's still broken.

syz51 commented 2 years ago

hey guys. do we have any progress on this issue? It is still happening on windows 10 at version 0.35

dcvlehr commented 2 years ago

happening to me on v0.36.1 as well (windows 11)

vyuranov commented 7 months ago

same in Windows 11 (terragrunt version v0.53.5)

philipp-durrer-jarowa commented 7 months ago

Same here terragrunt version v0.53.6 windows 11, used in mingw64 bash shell

rjdudley commented 6 months ago

edit: This is cured by setting the TERRAGRUNT_DOWNLOAD environment variable to a folder outside of your terragrunt code, per https://terragrunt.gruntwork.io/docs/features/caching/. :edit

Same issue, using Windows Terminal and PowerShell on Win 11. If I delete the cache folders, the error does not occur on the first "run-all plan", but does recur on subsequent runs. Moving the cache to a different folder does prevent the error, but adds to the command line (a config setting would be cool).

Additionally, if I just use terragrunt plan, I get a different error:

level=warning msg=Error removing manifest file C:\(redacted)\.terragrunt-cache\(redacted)\.terragrunt-module-manifest: remove C:\(redacted)\.terragrunt-cache\(redacted)\.terragrunt-module-manifest: The process cannot access the file because it is being used by another process. I can delete the cache subfolders in Windows explorer without issue, despite what the error says. This seems like something peculiar to the way Windows handles the cache folders.