antonbabenko
commented
3 years ago I think you are right that the solution you have implemented does not sound like the best one. You have a lot of extra implementation details (vendored modules, yaml files, etc) which I will skip and describe how this can be implemented in a simpler way (some details are ignored for simplicity reasons).
Terraform modules should be versioned. Combine multiple Terraform modules blocks into one (so-called, "infrastructure modules" or "stacks").
Terragrunt configuration describes environments that consist of versioned terraform module + inputs to that module.
Dependencies inside of terraform modules and in terragrunt configurations can be managed using tools like dependabot or renovatebot.
I don't see the reason to not use native mechanisms to resolve required dependencies implemented by terraform init in your scenario.
terraform and terragrunt does not have all the features like package management software has (think about npm, pip, etc), and in many cases, we don't need it.
https://github.com/antonbabenko/terragrunt-reference-architecture - take a look at this Terragrunt reference architecture.
I saw you touch on dependencies in https://github.com/antonbabenko/terraform-best-practices/blob/master/not-best-practices/faq.md but I was wondering if you could point to a resource for how to manage terraform dependencies..
My current organization does something that feels wrong, and requires lots of manual rebuilds.. When a terraform module depends on another module, it is fetched into the 'build' directory, with the dependent module name as a folder in the top level structure of the built module. It is then 'vendored' into the .tar that we produce and push to our artifact repository (across environments), and then exported from the artifact repo into each different environments git repo with an appropriate version tag.
We use terragrunt to fetch the version of the repo with all of it's dependencies vendored into it.;
What is the 'right' way to have multiple module dependency resolved in terragrunt? We already have a top level module_versions.yaml file that is used to index the version, but you can't just update dependent_module in that file and get it pulled in, unless the toplevel_module gets rebuilt, and pulls that version in.
Are there any good resources showing how to create/reference/build the right module structure and it's references? I guess what I want to find is what you would put in https://www.terraform-best-practices.com/examples/terragrunt, but it currently appears empty!