hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.41k stars 9.5k forks source link

Override module resource from outside module #27525

Open nitrocode opened 3 years ago

nitrocode commented 3 years ago

Terraform Version

Terraform v1.3.2
on darwin_amd64

But the version doesn't matter here.

Use Cases

Instead of having to wait for a module owner to update a resource in their tf module and instead of having to fork the module itself to make those changes, it would be nice to override the module resource from outside the module so you have full access to all of the resource-to-override's arguments and still use most of the feature set that comes from using the module itself.

Overriding a module with a local patch is similar to how you can use kustomize to use an upstream helm chart and apply a patch. A "kustomize for terraform" but built-in so we don't have to build the feature into yet another terraform wrapper like terragrunt, atmos, terramate, etc.

Attempted Solutions

  1. terraform init
  2. Use patch to apply a local patch to .terraform dir
  3. terraform plan or terraform apply

Proposal

Example hcl but could be different. This would be a patch file that could be applied to the module when it is initialized.

module "example" {
  source  = "some/terraform-module/aws"
  version = "1.0.0"

  patches = [
    "${path.module}/example.1.0.0.disable-sg.patch",
  ]

  input_one   = "input"
  input_two   = "input"
  input_three = "input"
  input_four  = "input"

  # ...
}

Then terraform init could download the module to the terraform data dir (.terraform) and apply the patch to the downloaded module.

References

Workaround

Using the patch command

# get into dir
cd components/terraform/ecr
terraform init
# backup file to patch
cp .terraform/modules/xyz/main.tf{,.orig}
# makes changes to file
vim .terraform/modules/xyz/main.tf
# create patch
diff -Naur .terraform/modules/xyz/main.tf.orig .terraform/modules/xyz/main.tf > 0-patch-terraform-dir-module-xyz-main.tf.patch
# revert
rm -rf .teraform/
# restart with init and run patch
terraform init
patch -p0 < 0-patch-terraform-dir-module-xyz-main.tf.patch
terraform plan
elvis2 commented 3 years ago

I second this request. What is the point of using 3rd provider modules if you can not override them? As the world begins to modernize their IaC stacks, using 3rd party modules to create a more common stack templated approach, this override feature becomes critical.

asaranglr commented 3 years ago

Any updates on this? Seems like a neat feature to have.

danielebra commented 3 years ago

Any updates on this issue? This feature would be beneficial for using established modules that are lacking an additional modification.

pkrolikowski commented 2 years ago

It's tempting to override resource attributes from outside the module, but it will brake "encapsulate configuration" promise. Very often modules are created to force user to create infrastructure by the rules defined by the module. Enabling overriding resource definition from outside is a bad thing in this case.

For now you can use override logic and yes, you have to fork but maintaining is much easier.

waltervargas commented 2 years ago

Is there any updates on this? can you provide a mechanism similar to the outputs, maybe an export keyword, so you allow the caller to override resources in the module.

Z4ck404 commented 1 year ago

Are there any updates about this ?

simoncpu commented 1 year ago

Dear Santa Claus,

I wish for the ability to override module resources from outside the modules too.

Thanks, Santa!

nitrocode commented 1 year ago

There is one way to do it now

  1. terraform init
  2. Use patch to apply a local patch to .terraform dir
  3. terraform plan or terraform apply
nitrocode commented 1 year ago

Overriding a module with a local patch is similar to how you can use kustomize to use an upstream helm chart and apply a patch.

We're basically asking for a kustomize for terraform but built-in so we don't have to build the feature into yet another terraform wrapper like terragrunt, atmos, terramate, etc.

simoncpu commented 1 year ago

This sounds interesting. I wish this would work on Terraform Cloud though...

simoncpu commented 1 year ago

My use case is that I reorganized my code into two modules. Both modules call cloudflare_zone on the same zone. I'm looking for a way to let the main code override the resources on both modules so that Terraform would treat it as a single resource.

benamazing commented 1 year ago

Yes, that would be great if terraform provides a feature like kustomize patch.

kderck commented 1 year ago

This would still be great.

Saurabhbatham commented 1 year ago

This feature is required, it will help to customize the module resources on which you don't have control.

jdavis-politico commented 1 year ago

Really hoping this gets implemented.

brannondorsey commented 1 year ago

There are many times where I've wished to reach for a feature like this. I recognize it can break some of the beauty of a well-crafted module encapsulation/abstraction, but it's not uncommon to find yourself in a situation where it just makes practical sense to be able to craft an override for a module you don't control. This is especially the case in scenarios where vendoring a fork of someone else's module actually introduces more complexity than a well crafted and commented override would.

elvis2 commented 1 year ago

It's tempting to override resource attributes from outside the module, but it will brake "encapsulate configuration" promise. Very often modules are created to force user to create infrastructure by the rules defined by the module. Enabling overriding resource definition from outside is a bad thing in this case.

For now you can use override logic and yes, you have to fork but maintaining is much easier.

Back in 2021, this override feature was not fully developed. Maybe that is no longer the case now...

Saying that, I just wanted to chime and and continue beating the drum that this is a needed feature.

sanjeevpandey19 commented 1 year ago

+1 for this feature

crw commented 1 year ago

Thanks for your interest in this issue! This is just a reminder to please avoid "+1" comments, and to use the upvote mechanism (click or add the 👍 emoji to the original post) to indicate your support for this issue. This helps avoid notification spam for issues with high numbers of participants while enabling the maintainers to prioritize issues. Thanks again for the feedback!

liamhelmer commented 1 year ago

Agreed, there's a lot of use cases that this would improve! Please add this in.

dror-g commented 1 day ago

Yea, don't hold your breath.. seems other similar issues close as not planned .

Since this feature would require significant work and be potentially dangerous to users, we have decided to not implement this feature at this time and will be closing the issue.

Originally posted by @ghost in https://github.com/opentofu/opentofu/issues/808#issuecomment-1968337057