search

gruntwork-io / terragrunt

Terragrunt is a flexible orchestration tool that allows Infrastructure as Code written in OpenTofu/Terraform to scale.
https://terragrunt.gruntwork.io/
MIT License
8.07k stars 981 forks source link

Issue with module path logging #3428

Closed PaulBurridge closed 1 month ago

PaulBurridge commented 1 month ago

Describe the bug

Since #3400 module path logged in Group 1 has and incorrect path listed, also initial INFO entries related to "Retrieved output from" are missing.

Steps To Reproduce

Using source (but also happens with compiled version)

go run main.go run-all init --terragrunt-non-interactive --terragrunt-working-dir /workspace/redacted/environment-definitions/nonprod/na/projects/redacted/dev-na/us-east-2/databricks

Log:

Group 1
- Module ./primary../../../../../../../../..

Group 2
- Module ./primary/catalogs
- Module ./primary/compute_pools
- Module ./primary/external_locations
- Module ./primary/sql_warehouses

17:48:45.727 STDOUT [primary../../../../../../../../..] terraform: Initializing the backend...
17:48:49.996 STDOUT [primary../../../../../../../../..] terraform: Initializing modules...
17:48:50.031 STDOUT [primary../../../../../../../../..] terraform: Initializing provider plugins...
17:48:50.031 STDOUT [primary../../../../../../../../..] terraform: - Reusing previous version of hashicorp/time from the dependency lock file
17:48:50.158 STDOUT [primary../../../../../../../../..] terraform: - Reusing previous version of databricks/databricks from the dependency lock file
17:48:51.252 STDOUT [primary../../../../../../../../..] terraform: - Reusing previous version of hashicorp/random from the dependency lock file
17:48:51.328 STDOUT [primary../../../../../../../../..] terraform: - Reusing previous version of hashicorp/aws from the dependency lock file
17:48:51.416 STDOUT [primary../../../../../../../../..] terraform: - Using previously-installed hashicorp/random v3.6.2
17:48:51.756 STDOUT [primary../../../../../../../../..] terraform: - Using previously-installed hashicorp/aws v5.65.0
17:48:51.768 STDOUT [primary../../../../../../../../..] terraform: - Using previously-installed hashicorp/time v0.12.0
17:48:51.799 STDOUT [primary../../../../../../../../..] terraform: - Using previously-installed databricks/databricks v1.51.0
17:48:51.799 STDOUT [primary../../../../../../../../..] terraform: Terraform has been successfully initialized!
17:48:51.799 STDOUT [primary../../../../../../../../..] terraform: You may now begin working with Terraform. Try running "terraform plan" to see
17:48:51.799 STDOUT [primary../../../../../../../../..] terraform: any changes that are required for your infrastructure. All Terraform commands
17:48:51.799 STDOUT [primary../../../../../../../../..] terraform: should now work.
17:48:51.800 STDOUT [primary../../../../../../../../..] terraform: If you ever set or change modules or backend configuration for Terraform,
17:48:51.800 STDOUT [primary../../../../../../../../..] terraform: rerun this command to reinitialize your working directory. If you forget, other
17:48:51.800 STDOUT [primary../../../../../../../../..] terraform: commands will detect it and remind you to do so if necessary.
17:48:57.111 STDOUT [primary/sql_warehouses] terraform: Initializing the backend...

Expected behavior

Log:

Group 1
- Module primary/workspace

Group 2
- Module primary/catalogs
- Module primary/compute_pools
- Module primary/external_locations
- Module primary/sql_warehouses

17:51:36.409 INFO   [../../../databricks/rbac/groups] Retrieved output from ../../../databricks/rbac/groups/terragrunt.hcl
17:52:22.475 INFO   [../../../../platform/dev-na/us-east-2/kms_keys] Retrieved output from ../../../../platform/dev-na/us-east-2/kms_keys/terragrunt.hcl
17:52:33.557 INFO   [../../../databricks/rbac/svc_principals/dev_na_tableau] Retrieved output from ../../../databricks/rbac/svc_principals/dev_na_tableau/terragrunt.hcl
17:52:37.358 INFO   [../../../databricks/metastores] Retrieved output from ../../../databricks/metastores/terragrunt.hcl
17:52:40.240 STDOUT [primary/workspace] terraform: Initializing the backend...
17:52:43.879 STDOUT [primary/workspace] terraform: Initializing modules...
17:52:43.903 STDOUT [primary/workspace] terraform: Initializing provider plugins...
17:52:43.903 STDOUT [primary/workspace] terraform: - Reusing previous version of hashicorp/aws from the dependency lock file
17:52:44.040 STDOUT [primary/workspace] terraform: - Reusing previous version of hashicorp/time from the dependency lock file
17:52:45.169 STDOUT [primary/workspace] terraform: - Reusing previous version of databricks/databricks from the dependency lock file
17:52:45.250 STDOUT [primary/workspace] terraform: - Reusing previous version of hashicorp/random from the dependency lock file
17:52:45.631 STDOUT [primary/workspace] terraform: - Using previously-installed hashicorp/aws v5.65.0
17:52:45.639 STDOUT [primary/workspace] terraform: - Using previously-installed hashicorp/time v0.12.0
17:52:45.668 STDOUT [primary/workspace] terraform: - Using previously-installed databricks/databricks v1.51.0
17:52:45.677 STDOUT [primary/workspace] terraform: - Using previously-installed hashicorp/random v3.6.2
17:52:45.677 STDOUT [primary/workspace] terraform: Terraform has been successfully initialized!
17:52:45.677 STDOUT [primary/workspace] terraform: You may now begin working with Terraform. Try running "terraform plan" to see
17:52:45.677 STDOUT [primary/workspace] terraform: any changes that are required for your infrastructure. All Terraform commands
17:52:45.677 STDOUT [primary/workspace] terraform: should now work.
17:52:45.677 STDOUT [primary/workspace] terraform: If you ever set or change modules or backend configuration for Terraform,
17:52:45.677 STDOUT [primary/workspace] terraform: rerun this command to reinitialize your working directory. If you forget, other
17:52:45.677 STDOUT [primary/workspace] terraform: commands will detect it and remind you to do so if necessary.
17:52:49.200 INFO   [primary/workspace] Retrieved output from primary/workspace/terragrunt.hcl
17:52:51.101 STDOUT [primary/catalogs] terraform: Initializing the backend...

Versions

Additional context

Commits before #3400 and version 0.67.4 work as expected.

PaulBurridge commented 1 month ago

@levkohimins Happy to test a proposed fix when you have one, let me know if you need any more details.

levkohimins commented 1 month ago

@PaulBurridge, could you please provide a sample config to reproduce this issue?

PaulBurridge commented 1 month ago

@levkohimins Example repo here: https://github.com/PAB-me-uk/terragrunt-bug-3428

denis256 commented 1 month ago

It is interesting that in my environment (Ubuntu 24.04.1 LTS) it works fine...

terragrunt run-all init --terragrunt-non-interactive --terragrunt-working-dir ./bad
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.67.5
17:48:06.478 INFO   The stack at ./bad will be processed in the following order for command init:
Group 1
- Module ./bbb/ccc/workspace

Group 2
- Module ./bbb/ccc/module-b

17:48:06.538 STDOUT [bbb/ccc/workspace] tofu: Initializing the backend...
17:48:06.578 STDOUT [bbb/ccc/workspace] tofu: Initializing provider plugins...
17:48:06.578 STDOUT [bbb/ccc/workspace] tofu: - Reusing previous version of hashicorp/null from the dependency lock file
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: - Using previously-installed hashicorp/null v3.2.3
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: OpenTofu has been successfully initialized!
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: 
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: any changes that are required for your infrastructure. All OpenTofu commands
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: should now work.
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: If you ever set or change modules or backend configuration for OpenTofu,
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: rerun this command to reinitialize your working directory. If you forget, other
17:48:06.695 STDOUT [bbb/ccc/workspace] tofu: commands will detect it and remind you to do so if necessary.
17:48:06.757 STDOUT [bbb/ccc/module-b] tofu: Initializing the backend...
17:48:06.797 STDOUT [bbb/ccc/module-b] tofu: Initializing provider plugins...
17:48:06.797 STDOUT [bbb/ccc/module-b] tofu: - Reusing previous version of hashicorp/null from the dependency lock file
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: - Using previously-installed hashicorp/null v3.2.3
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: OpenTofu has been successfully initialized!
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: 
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: any changes that are required for your infrastructure. All OpenTofu commands
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: should now work.
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: If you ever set or change modules or backend configuration for OpenTofu,
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: rerun this command to reinitialize your working directory. If you forget, other
17:48:06.930 STDOUT [bbb/ccc/module-b] tofu: commands will detect it and remind you to do so if necessary.
levkohimins commented 1 month ago

@PaulBurridge, thanks for the sample. Just tried it on MacOS, Terragrunt version 0.67.8, works fine, same output as @denis256. @PaulBurridge perhaps you missed something in the sample. What is your OS?

levkohimins commented 1 month ago

Although I have noticed one minor bug: file:///workspace/calypso/env-def-test is converted to file://...

@PaulBurridge, It is also important for me to know the full path to the sample directory on your OS, this is important for accurate reproduction.

PaulBurridge commented 1 month ago

Running with absolute path

terragrunt run-all init --terragrunt-non-interactive --terragrunt-working-dir /workspace/test/terragrunt-bug-3428/bad
08:32:04.612 INFO   The stack at . will be processed in the following order for command init:
Group 1
- Module ./bbb/ccc../../..

Group 2
- Module ./bbb/ccc/module-b

Copied into /tmp

terragrunt run-all init --terragrunt-non-interactive --terragrunt-working-dir /tmp/terragrunt-bug-3428/bad 
08:32:15.651 INFO   The stack at . will be processed in the following order for command init:
Group 1
- Module ./bbb/ccc/workspace

In dev container

cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm

I was wondering what was special about the module being called called workspace so I assume this is related to the root directory being also called workspace?

levkohimins commented 1 month ago

I was wondering what was special about the module being called called workspace so I assume this is related to the root directory being also called workspace?

Correct. I was able to reproduce this behavior. I will fix it soon. Thanks for your cooperation.

levkohimins commented 1 month ago

Resolved in v0.67.12 release.