brikis98
commented
4 years ago @yorinasub17 Could you take a look when you're back?
Open chandrakanthkannam opened 4 years ago
brikis98
commented
4 years ago @yorinasub17 Could you take a look when you're back?
yorinasub17
commented
4 years ago Hi can you share the contents of database/terragrunt.hcl and common.hcl?
Based on the stack trace, it appears that the HCL parser is cycling when it tries to parse out the config when it tries to read it in. It doesn't make sense to me yet why it would behave differently depending on where it is placed, although I have a few hypotheses. Either way, it is hard to evaluate what might be going on without seeing the config it is trying to read in, since the failure is in the config that is read in and not what is referenced.
chandrakanthkannam
commented
4 years ago Hi @yorinasub17 ,
Sure below are the contents of database/terragrunt.hcl
terraform{
source = "<source>"
}
include{
path = find_in_parent_folders()
}
dependency "vpc" {
config_path = "../vpc"
}
dependency "ad" {
config_path = "../ad"
}
dependency "s3" {
config_path = "../s3"
}
dependency "scripts"{
config_path = "../scripts"
}
inputs = {
cluster_name = "cluste_name"
witness_instance_type = "t3.small"
witness_ami_id = "ami-"
database_instance_type = "t3.medium"
database_ami_id = "ami-"
domain_name = dependency.ad.outputs.ad_domain
domain_netbios = dependency.ad.outputs.ad_netbios
domain_secrets = "<secrets-name>" //these needs to be created
sql_secrets = "<secrets-name>"//these needs to be created
subnets = dependency.vpc.outputs.private_subnets
config_bucket = dependency.s3.outputs.envBucketName
config_bucket_prefix = "scripts"
database_disks = {<string>}
disk_initializer = dependency.scripts.outputs.scripts.name
vpc_id = dependency.vpc.outputs.vpc_id
}and here is the common.hcl:
inputs = {
cluster_name = "cluste_name"
witness_instance_type = "t3.small"
witness_ami_id = "ami-"
database_instance_type = "t3.medium"
database_ami_id = "ami-"
}Could this because the terragrunt.hcl in database module has inputs which are dependent on some other modules?
yorinasub17
commented
4 years ago Thanks for sharing that! However, as you suggested, I think I need to keep seeing up the dependency tree 😅
It does look like the HCL parser is blowing up on reading one of the dependencies, and not the database config. The reason is that the stack trace suggests a blow up in the HCL parser related to an expression that:
+, -, && etc)local references?)Is there a config in your dependency tree that matches this criteria? If so, can you share that so I can get a sense of how deep the expressions go and file the necessary issue to the HCL parser?
Also, is terragrunt able to manage the database config by itself? If so, this is unlikely, but then possibly the extra calls added to the stack from going through read_terragrunt_config might be what caused the stack overflow and you were already close to the limit in the config.
Thanks!
chandrakanthkannam
commented
4 years ago Yes @yorinasub17, in the database config dependencies one of them has condition in their inputs and here is that file:
terraform{
source = "<source>"
}
include{
path = find_in_parent_folders()
}
dependency "vpc" {
config_path = "../vpc"
}
inputs = {
vpc_id = dependency.vpc.outputs.vpc_id
subnet_ids = length(dependency.vpc.outputs.private_subnets) >= 3 ? ["${element(dependency.vpc.outputs.private_subnets,1)}","${element(dependency.vpc.outputs.private_subnets,2)}"] : dependency.vpc.outputs.private_subnets
mgmt_public_subnet_id = length(dependency.vpc.outputs.private_subnets) >= 1 ? ["${element(dependency.vpc.outputs.private_subnets,0)}"] : dependency.vpc.outputs.private_subnets
mgmt_private_subnet_id = length(dependency.vpc.outputs.private_subnets) >= 1 ? ["${element(dependency.vpc.outputs.private_subnets,0)}"] : dependency.vpc.outputs.private_subnets
mgmt_public_ip = true
domain_admin = "<username>"
ssm_parameter_store_name = "<store_name>"
}Also, is terragrunt able to manage the database config by itself? ---> Yes, if i apply the database config file it's going through with out any error.
yorinasub17
commented
4 years ago Thanks for your response. I still need to dig deeper into this, but so far I haven't found anything that is causing this or been able to repro.
With that said, a quick sanity check: is it possible that you have an accidental cyclic dependency? The reason is that I was looking at the log again and it looks like:
read_terragrunt_config => dependency => read_terragrunt_config => dependency ...Since I can't see the second read_terragrunt_config, I was just wondering if you had accidentally cycled. It would also help knowing how deep your links are: how many chains of dependencies and read_terragrunt_config do you have?
amasover
commented
4 years ago I see something similar, with a setup like this:
(the parent) environment/terragrunt.hcl:
locals {
child_config = read_terragrunt_config("${get_terragrunt_dir()}/terragrunt.hcl")
}
terraform {
before_hook "child-config" {
commands = ["init"]
execute = ["bash", "-c", "echo ${local.child_config}"]
}
}(the child) environment/module/terragrunt.hcl:
terraform {
source = "git@somegitrepository//mymodule"
}
include {
path = find_in_parent_folders()
}Seems like maybe a cyclic dependency of child -> parent -> child -> parent ?
My use case is to know the value of source from the child terragrunt.hcl without parsing it out myself.
yorinasub17
commented
4 years ago Ah yes that is not supported. The infinite loop is:
include.read_terragrunt_configread_terragrunt_config parses childSince we aren't doing anything fancy with the parsing (e.g., partial parsing and recursing), this can't be handled properly.
We should definitely have cycle detection with read_terragrunt_config like dependency blocks though. I am a bit buried at the moment and most likely will not get to anything like that soon.
My use case is to know the value of source from the child terragrunt.hcl without parsing it out myself.
This is probably not a use case we are going to support any time soon, since you basically want a global namespace between the parent and the child for ALL attributes (not just globals, a theoretical block that was discussed in the past, or locals). You will most likely have to wait for https://github.com/gruntwork-io/terragrunt/issues/759 to be implemented.
That said, what do you want to do with the source? There might be other workarounds that break the cycle.
amasover
commented
4 years ago This is probably not a use case we are going to support any time soon
No worries, I think I can manage for now :)
That said, what do you want to do with the source? There might be other workarounds that break the cycle.
I have some hooks that I'd like to run for all modules except one. The module folder name in my Terragrunt environment structure can vary. For example, it could be either of these:
environment/module/terragrunt.hcl
environment2/module-with-different-name/terragrunt.hclBut the source would be the same for both of those. So I can't use get_terragrunt_dir() to parse the path, but if I know what's past the // in source, then I can decide whether or not my hook should fire.
If there are other workarounds, or a better way to do things, I definitely appreciate the help.
Hi,
I was trying to use the new function
read_terragrunt_configto read inputs from a different child module instead of the config file in parent location and present .hcl file looked like this Test 1:Note: I tried this syntax in locals block
db_vars = read_terragrunt_config("../database/terragrunt.hcl")and its throwing the same results as mentioned in Results GotExpected Result: It to import the inputs from
../database/terragrunt.hclto the present child moduleResults Got: While doing a terragrunt plan-all the process clocking high memory usage on my machine and throwing this result after few minutes of blank screen:
Test 2: I placed the test config file in parent directory and tried to read that using the function, the code block looks like this
Expected Result: It to import the inputs from common.hcl Results Got: It worked as expected
Could you please help me here understanding why my Test 1 is failing , is it not a valid way to use the function
read_terragrunt_config?