coretech / terrafile

A binary written in Go to systematically manage external modules from Github for use in Terraform
Apache License 2.0
141 stars 28 forks source link

Nested Modules? #1

Open osterman opened 5 years ago

osterman commented 5 years ago

First off, this is sweet! We are looking for a way to vendor terraform modules and this looks like it could be the best way. It’s clear to me how this works.

  1. Does it support the case of automatically rewriting the source parameter in nested modules? E.g. modules that call other modules.
  2. If not, would this be practical or in scope for this project?
nritholtz commented 5 years ago

Hey @osterman, glad you like it! We're using this internally and it really does make our terraform module management alot easier. Sorry that the documentation is poor, we have plans to eventually make a clearer README, with additional test coverage which should really show all the features that this tool can do. PR's are always welcome! 😄

I think I understand your use case so correct me if I'm wrong:

*terrafile*
foo1:
    source: "git@github.com:foo/foo1"
    version: "v1.0"

*main.tf*
module "service" {
source = "./vendor/modules/foo1//aws/service"
....
}

*foo1 - terrafile*
foo2:
    source: "git@github.com:foo/foo2"
    version: "v1.5"

*foo1 - aws/service*
module "another_service" {
source = "./vendor/modules/foo2//aws/another_service"
....
}

*foo2 - aws/another_service*
resource "aws_autoscaling_group" "example" {
......
}

I have not fallen into this use case yet, as most of my own module are self contained or using public modules that are not using Terrafile. However, I can definitely see the benefit and would be totally open to discuss the possibility of us adding this functionality (or accepting PRs!) My only hesitation is that the test coverage on this is very minimal at the moment and would love to get it higher before adding even more complexity to the tool.

A possible workaround as of right now off the top of my head (so if I'm wrong, please blame the fact I'm on my first coffee) is to run terrafile again within each of the newly cloned module repos, but I understand that this might not be feasible.

osterman commented 5 years ago

@nritholtz yea, I think we're on the same page.

For some additional context, we maintain 120+ free terraform modules on our github. One of our core modules is terraform-null-label which we use to generate consistent resource names. As such, every single module we write uses this module inside of it. For example, our terraform-aws-eks-cluster module uses it.

So what we want to do is to basically use terrafile to bundle all modules locally for the purpose of vendoring. This allows users to "rest assured" that they have full-control over what is executed in their environment.