gruntwork-io / terragrunt

Terragrunt is a flexible orchestration tool that allows Infrastructure as Code written in OpenTofu/Terraform to scale.
https://terragrunt.gruntwork.io/
MIT License
8.06k stars 977 forks source link

include block to load terrgrunt.hcl from remote zip #1070

Open sleungcy opened 4 years ago

sleungcy commented 4 years ago

is it possible in the include block to load the root terragrunt.hcl from a remote zip file like the module source parameter for Terraform module blocks?

I am looking into making all my application deployments decentralized, such that developers who are deploying a single application during development do not need to perform extra steps to download the repo holding all the terragrunt.hcl files.

yorinasub17 commented 4 years ago

This is an interesting idea and I can see this being useful for improving terragrunt support in a poly repo setup. read_terragrunt_config, dependency, and the upcoming import block would probably also benefit from having something like that.

With that said, I am not sure we would want to maintain this going forward since we recommend a monorepo setup for the canonical use of terragrunt. This largely depends on if there is a sane way to maintain this in the increasingly complex terragrunt parsing code.

If anyone is interested in implementing this, the next steps would be to propose a detailed RFC that includes:

sleungcy commented 4 years ago

I've been able to achieve this polyrepo setup without additional features from terragrunt.

1) Using the --terragrunt-source cli argument to download and extract root level terragrunt.hcl. 2) In my application terragrunt.hcl, use the fileset function to search for the terragrunt.hcl from .terragrunt-cache. e.g.

terragrunt.hcl
include {
  path = fileset( ".terragrunt-cache", "**/root-terragrunt.hcl" )
}

The project stucture will look like:

App#1

App#2

The reason I am doing this, is that we have 100+ microservices, and different teams work on only a small subset of these Apps. If we go with monorepo approach, our devs will have to first download the monorepo, which has 100+ subdirectories containing all the apps, and then using include/exclude filters to deploy their app. For developer experience and CD workflows, I find that reversing the dependency makes it a lot more seamless.

Perhaps my use case is quite different from the majority, as we are using terraform to deploy microservices rather than infrastructure.

pckbls commented 3 years ago

Another workaround would be to make use of Terragrunt's run_cmd command, i.e.:

include {
  path = run_cmd("sh", "-c", "curl -s https://PATH/TO/terragrunt.hcl -o ../terragrunt.hcl && echo ../terragrunt.hcl")
}