feature request: inverse targeting / exclude

[shubhambhartiya]

Is there anything that can be done such that db_instance - RDS formed by the terraform files can be saved if we destroy the whole state?

[phinze]

Hi @shubhambhartiya - we have prevent_destroy which provides protection against accidental destruction, but it sounds like perhaps you're asking about "destroy everything but this" feature.

Can you elaborate on the behavior you're looking for?

[shubhambhartiya]

Considering an example. I have a set of tf files which creates vpc, subnets, ASG, sg, instances in various subnets, nat instances and databases (RDS). I want to plan in this way such that when I destroy the plan, I want the RDS to be there (VPC and subnets would be needed), rest all the things would get destroy.

[phinze]

Ah okay I get it now. I think I'd call what you're looking for "inverse targeting".

# Destroy everything except aws_db_instance.foo and its dependencies
terraform plan -destroy -exclude=aws_db_instance.foo

^^ If that looks like what you're asking for I'll edit the title and we can track that feature request with this thread.

[shubhambhartiya]

Yes, that would be a great thing.

[glerchundi]

yes please! :+1:

[josephholsten]

this is related to #4515

[anosulchik]

Just to confirm that it would be nice to have a feature of inverted targeting like as follows:

terraform apply -target-exclude aws_ecs_service.ecs_service

Thanks.

[anagrius]

+1

[analytically]

+1

[beanaroo]

Another use case:

We're importing existing AWS environments. Migrating the DB to the new subnet group is a manual step. It would be nice to provision all the subnet/security/parameter groups before updating the instance (all part of the same module)

[cmacrae]

I'd also love to see this. In the meantime, I'm using a combination of lifecycle to "protect" certain resources, and targeting like so:

terraform plan -destroy $(for r in `terraform state list | fgrep -v resource.address.to.exclude` ; do printf "-target ${r} "; done) -out destroy.plan

Not pretty, but it does the job 🙂

[olenm]

Following what @anosulchik posted:

Just to confirm that it would be nice to have a feature of inverted targeting like as follows:terraform apply -target-exclude aws_ecs_service.ecs_service.

A target and something like a -target-exclude would be great to support regexp or by name-matching similar to consul, such as: terraform apply -target-exclude aws_ecs_service. would match all that start with aws_ecs_service. or if its regexp it can be more explicit which would be ideal

thanks

[ffoysal]

terraform destroy -target-exclude aws_db_instance.my_rds It would be great to have this feature. so that we can destroy everything except rds instance. It will save a significant amount of time for us if we can just destroy everything except rds resource, as rds takes around 30 minutes to create and timeout during destroy

[ColOfAbRiX]

This would be really useful, so I can destroy everything except the resources marked with prevent_destroy. At the moment, because of prevent_destroy, I comment out everything except that code and run apply instead of destroy. Very unintuitive.

[henning]

+1 :)

My current workaround is to taket the outout of "terraform plan list" , grep out all resource I wanna keep, and then create a list of -target parameters from the rest with a shell script.

Another thing that would make it supereasy to destroy everything unless the things you want to keep is to destroy all resources instead of those protected by the "prevent_destroy" flag. Actually, in my opinion the behaviout for that flag is not ideal - if I call destroy, I want to destroy the configured resources, and to me it's somehow logical that the prevent_destroy flag is only for the one resource, not for the whole setup. The way it works now it protects the whole configuration, not the single resource from destruction. And this is the most mentioned reason why people need this here...

[laura-herrera]

Still, it would be very useful to have terraform apply -exclude as sometimes your ECS cluster has changed due to Autoscaling rules and you don't want to change that, but might want to add more resources, etc

[idjaw]

Jumping in to provide a use case I'm trying to meet in which I'm trying to run my tf file but exclude just one resource that calls ansible, which for testing purposes do not want to worry about for the time being. Right now it seems like I have to do several "-target" to include what I want, and that takes a very long time considering how many resources I have in my module.

So, to re-iterate what someone else suggested, something like:

terraform plan -target-exclude="" -target-exclude=""

Ideally, it would be awesome to provide a list:

terraform plan -target-exclude="item1 item2 item3"

[jonvines]

+1

[josephholsten]

Please refrain from commenting on this ticket unless you have something to help complete this ticket.

Speaking of which, could anyone review https://github.com/hashicorp/terraform/pull/3366 and let me know what’s remaining?

[royerhq]

+1

[olenm]

@josephholsten https://github.com/hashicorp/terraform/pull/3366 looks great! looking forward to this, I will test run your code next week on my env and report back

[josefeg]

+1

[JorgeAlmeida]

+1

[jpdoria]

Hello, any updates on this?

[creativeux]

+1

[tdmalone]

@creativeux & others, please refrain from posting +1 comments or other comments that don’t add to the discussion. It increases noise for everyone subscribed to the issue. Use the emoji reactions instead.

[zhao-li]

I found this thread because I was learning Terraform and thought I needed this capability. But I found a workaround that no longer required me to need exclude. Hopefully, it will help others that might be in the same boat as me and help them find their way around Terraform.

If I restructured my configurations, I am able to use data instead of resource, which then allowed me to destroy all of my resources without triggering Terraform to destroy any resource that is managed elsewhere.

So for example, I initially was using:

# terraform-configs/main.tf
resource "aws_ecs_cluster" "my_cluster" {
  cluster_name = "MyCluster"
}

and this told Terraform to manage this cluster and, hence, destroy it when terraform destroy terraform-configs/ is issued.

Then I found that I can move the cluster configuration to another folder so I can still use Terraform to create/destroy/manage it from there, but access it using data instead:

# terraform-configs/transient-stuff/main.tf
data "aws_ecs_cluster" "my_cluster" {
  cluster_name = "MyCluster"
}
# terraform-configs/persistent-stuff/main.tf
resource "aws_ecs_cluster" "my_cluster" {
  cluster_name = "MyCluster"
  ...
}

So now, I am able to issue terraform destroy terraform-configs/transient-stuff/ and not worry about the cluster getting blown away. If I do need to create/destroy/manage the cluster, I can then do terraform destroy terraform-configs/persistent-stuff/.

I'm a newbie at Terraform, so I'm sure there might be an even better way that I am missing. But at least this might avoid needing exclude for now.

Hope that helps :)

[giuliov]

I have an additional scenario to share.

I have trouble with the new configuration of a VM. So I manually disconnected a public IP address using Azure Portal and do not want it to be reconnected when I recreate the machine, but at a later stage when I am happy with my tests and can be sure not to create a security hole.

[ahl]

It would be extremely useful to us to have this; any chance it will be prioritized?

[emmm-dee]

Another +1

Prepping for a migration. I am using a shared module between environment declarations. One component of the module writes public DNS records upon ELB creation. I want this to happen for staging/dev but "not-quite-yet" for production since prod is currently pointed to the current prod env as an A record. New record post-migration is an ALIAS record root module that's being called so I can't just set the current A root domain record to the current value. Sure I could add some conditional logic and add more variables and mess with the module declaration to choose between standard and alias records ugh. What I'm doing is include everything EXCEPT the one dns module that I 'don't quite' want to apply yet in the pre-migration prod environment. This is only temporary but it's hindered workflow for now. Would be so convenient to be able to just exclude that one module instead of include on others that are pending changes.

It just seems logical in software that anywhere you can include something you should also be able to negate it with an exclude in some way.

[mildwonkey]

Please do not post "+1" comments here, since it creates noise for others watching the issue and ultimately doesn't influence our prioritization because we can't actually report on comments. Instead, react to the original issue comment with 👍, which we can and do report on during prioritization.

[franherrera3112]

We would be glad to use this as it will be a function that prevents specific targets to be destroyed for example dev environments that need to be on for more than 8 hours a day

[saiprasannasastry]

any update on this feature?

[mercuriete]

This is happening to me:

1. I want to prevent_destroy a ip address.
2. I want destroy all but resources with "prevent_destroy"
3. I have this error.

   terraform destroy
   Resource google_compute_address.tb-vm-ip has lifecycle.prevent_destroy set,
   but the plan calls for this resource to be destroyed. To avoid this error and
   continue with the plan, either disable lifecycle.prevent_destroy or reduce the
   scope of the plan using the -target flag.

Solution: Change or extend terraform destroy to destroy all but prevent_destroy resources. With actual behaviour prevent_destroy is almost useless

Sorry for make spam, my request is: https://github.com/hashicorp/terraform/issues/3874

[Dzhuneyt]

4 years after this request was acknowledged to be useful and still no movement?

[anupkulkarni24]

this is strange , this feature is required by so many people, has it not been prioritized , Please help to add this feature as we are using terraform heavily and are now blocked because of this :(

[whereisaaron]

@anubhavmishra there are several open issues all circling the same core Terraform problem for 4-5 years, with hundreds of 👍, many subscribers, many reports and use cases. But there seems no will to address it.

• feature request to extend/change lifecycle.prevent_destroy #2159
• feature request: inverse targeting / exclude #2253
• prevent_destroy should let you succeed #3874
• Persistent Resources #20065

This issue and state management was such drama, that in the end we went back to CloudFormation, which had improved quite a bit in the meantime.

[davidlu1001]

Due to terraform doesn't support the feature yet, so would like to provide a wrapper to implement fake exclude/include feature, using Makefile and Terraform resource targeting.

It might be useful if there's too many unrelated resources and you don't want to copy & paste over 50 times for the targets.

Yeah ugly, but works (hopefully) 😅

Really look forward to official support for this feature.

---

Before:

e.g.

PLAN_OPTIONS="-target="A" -target="B" ... -target="N"" terraform plan

After:

e.g.

make plan_exclude EXCLUDE='aws_autoscaling_group|aws_launch_configuration'
make plan_include INCLUDE='aws_security_group|iam'

The basic idea is to run make plan first to show pending changes, also generate current.plan that we use later to filter targets.

And then to filter the resources we want by using [INCLUDE | EXCLUDE] which support POSIX Extended Regular Expressions (ERE).

Finally, use make apply directly to apply the changes based on new current.plan.

Gist could be found here:

https://gist.github.com/davidlu1001/e832038299fff99d4a4b2c6a75d71b78

[jjorissen52]

@davidlu1001 (and everyone suggesting faking exclusion by explicitly constructing a target list that excludes the desired resources) Unfortunately, when your target list becomes very long (49, in my case) You start to see Too many command line arguments. Configuration path expected., so this workaround is not enough.

[davidpasqua]

any update on this feature? It will be great to have a way to exclude --exclude or .tfignore or similar thanks in advance

[fumantsu]

I'm looking also for this feature. I want to prevent IAM roles for example for any test I'm doing.

[007]

Seconded what @fumantsu needs, if I'm managing a full stack with a single terraform state I'd like to be able to terraform plan -out plan.out -target-exclude module.iam to get everything applied but not require an admin user with extended IAM permissions. Same goes for @shubhambhartiya's use-case, with -destroy it'd be nice to keep something like a DB/subnet intact between runs of rebuilding everything around it. Individual -target rules can't work when you have a few hundred resources that make up a single stack, you'll inevitably run out of commandline space or miss out on applying something if it's not automated and built-in.

This could apply to other things like the helm or postgres providers, where I may be managing the infrastructure, but not have a grant for the k8s cluster or PG database themselves.

[tehmaspc]

I've been following this thread for a while now and respectfully IMO I feel that if folks are composing their dependencies properly they shouldn't need to be doing this or can avoid some the problems listed in this thread. For other examples it seems that folks are expecting a tool that manages state to keep working when they modify state outside of said tool.

One does need to appreciate the difficult task of building a state management tool that could then ad-hoc allow folks to negate certain state at their whim. Though, the --target feature can make someone assume that this would be an easy ask. I personally would be surprised to ever see this come to fruition.

I definitely commiserate w/ the situations where one has abandoned infrastructure or you are the unlucky sole that needs to manage someone else's prior mistake - :(

[stuart-c-moore]

Sorry @tehmaspc but you can't just assume everyone else is "doing it wrong".

Having a terraform environment of many hundreds of recourses, but only wanting to destroy some of them for a given task is a perfectly valid use case.

[mercuriete]

@tehmaspc ok, supose that you have a continous integration pipeline that apply a testing environment to do real test and not do simple terraform plan. then you CI destroys everything but a single resource like a address ip.

then a devops adds a count in a VM. that implies a change in the name of the resource.

your approach doing -target fails terribly because the new name. and then you have to update the CI pipeline to use a new -target.

but then everything is unusable because we cant deploy a commit before or after that because CI doesn't work properly and you have to delete objects manually (breaking change).

I think what @stuart-c-moore said is right.

there are plenty of use cases that you want to destroy everything but one resource. (inverse target / exclude)

[emmm-dee]

@anubhavmishra there are several open issues all circling the same core Terraform problem for 4-5 years, with hundreds of 👍, many subscribers, many reports and use cases. But there seems no will to address it.

• feature request to extend/change lifecycle.prevent_destroy #2159
• feature request: inverse targeting / exclude #2253
• prevent_destroy should let you succeed #3874
• Persistent Resources #20065

This issue and state management was such drama, that in the end we went back to CloudFormation, which had improved quite a bit in the meantime.

Ugh, you're right, CloudFormation is really seeming more and more enticing even though I really don't want to be dealing with a bunch of JSON files. It's just becoming a bit unrealistic to stick with TF as AWS feature sets seems to be growing and evolving much faster than TF feature set. It was good while it lasted and can't complain since it's free software.

[berkil]

5 years after the feature was requested, I guess that it's not yet added ?

[zeknox]

I echo everyone else's sentiment in the thread. This would be an incredibly valuable feature.

[aldarisbm]

This becomes necessary to manage more complex infrastructures

[Tbohunek]

"How ward could it be..." Jeremy Clarkson