Support use cases with conditional logic

[phinze]

It's been important from the beginning that Terraform's configuration language is declarative, which has meant that the core team has intentionally avoided adding flow-control statements like conditionals and loops to the language.

But in the real world, there are still plenty of perfectly reasonable scenarios that are difficult to express in the current version of Terraform without copious amounts of duplication because of the lack of conditionals. We'd like Terraform to support these use cases one way or another.

I'm opening this issue to collect some real-world example where, as a config author, it seems like an if statement would really make things easier.

Using these examples, we'll play around with different ideas to improve the tools Terraform provides to the config author in these scenarios.

So please feel free to chime in with some specific examples - ideally with with blocks of Terraform configuration included. If you've got ideas for syntax or config language features that could form a solution, those are welcome here too.

(No need to respond with just "+1" / :+1: on this thread, since it's an issue we're already aware is important.)

[lubars]

@serialseb, how can I apply this technique to an ebs_block_device? It doesn't have a countparameter.

[pll]

@lubars - I believe count is a meta param and can be applied to anything.

[jasonf20]

It can only be applied to resources, which somewhat limits it's use as a "when" condition On May 9, 2016 22:39, "Paul" notifications@github.com wrote:

@lubars https://github.com/lubars - I believe count is a meta param and can be applied to anything.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/hashicorp/terraform/issues/1604#issuecomment-217966782

[pll]

Oh, ebs_block_device. I think my brain interpreted that as aws_ebs_volume... Sorry about that.

[serialseb]

You could just create ebs devices and use attachments if you want to use count? Had some issues with doing that with root devices though.

[lubars]

I thought about that - I guess I'd add a count to both the aws_ebs_volume and aws_volume_attachment and make sure they match?

And I've seen the admonition against mixing ebs_block_device with aws_ebs_volume; is root_block_devicea kind of ebs_block_device(and could that be the source of the issues you encountered)?

[serialseb]

The count would work, order is guaranteed so 0..n volumes, 0..n attachments and they'll match, you element(aws_ebs_volume.*.id,count.index) and off you go. You probably will want the device name to come from a list too as those are required.

I've not investigated that much but i tried having the aws_ebs_volume from a snapshot and an attachment, using the root device name, and it failed on me because the ami already had the volume mount on it. In normal scenarios it may well just work, untested.

[ketzacoatl]

Please correct me if I am wrong, but I believe the problem with using volumes attached outside of the aws_instance block is that if you want to run init on the instance you've created, you don't have any guarantees for when the volumes are actually attached, and there is potential for race conditions or bad assumptions during init.

In short, there are times when you really need to define an EBS volume inline with aws_instance, and using count is not always available or reasonable.

[serialseb]

Maybe we're kinda polluting this github issue with this conversation? My fault, I'm sorry :/ i think terraform will always wait for the instance to be up before attaching the drive for aws_instance, for asg you can configure that, scripts i tend to loop till device becomes avail

[lubars]

@ketzacoatl I had this exact problem with Azure (classic) using azure_data_disk- couldn't get disk creation to precede instance creation - even using a [depends_on] clause.

[franklinwise]

@pll - the example I gave was to focus on the syntax, rather the specific case of creating a machine or not.

Let's say I want a different hard drive configuration for production.

Dev: Standard EBS Volume

Prod: Multiple Volumes

resource "aws_volume_attachment" "ebs_att" {
  device_name = "/dev/sdh"
  volume_id = "${aws_ebs_volume.example.id}"
  instance_id = "${aws_instance.web.id}"
  when = "${var.env == prod}"
}

resource "aws_instance" "web" {
  ami = "ami-21f78e11"
  availability_zone = "us-west-2a"
  instance_type = "t1.micro"
  tags {
    Name = "HelloWorld"
  }
}

resource "aws_ebs_volume" "example" {
  availability_zone = "us-west-2a"
  size = 1
  when = "${var.env == prod}"
}

[ajlanghorn]

Our infrastructure makes use of public and private load balancers in AWS. Public-facing applications (be they browsers or other systems away from our infrastructure) make use of the public load balancers, and applications inside our infrastructure make use of private load balancers to talk to each other. Some applications, as a result, have both a public and a private load balancer, but some do not.

We were trying to come up with a way to only add private load balancers to autoscaling groups if they exist. A conditional would have been great here, but alas we ended up with this:

load_balancers = ["${compact(split(",", concat(aws_elb.elb-public.name, ",", var.private_elb_name)))}"]

where aws_elb.elb-public is from a reusable module, so that just works, and var.private_elb_name gets the name of the private load balancer to attach to the ASG during the instantiation of the module.

[ketzacoatl]

I have a TF module that creates a security group for a nomad agent. If there was a conditional available such that I could do the equivalent of {% if open_ephemeral_ports %}, it would allow that one module to add the 20000 to 60000 range to the security group, if the user-developer wants that, rather than having to create a second module that is nearly identical (and making a user-developer choose between multiple modules).

[blakeneyops]

I've expanded on the use of count to implement conditional logic a bit that some of you may find useful. I've always been unhappy with the need to duplicate code to model different options on the same type of resource. It makes it very difficult to create reusable generic modules. Additions of resources like aws_security_group_rule and aws_route have started to help this a lot, but there are still plenty of scenarios where it is a problem. One such scenario would be configuring an autoscaling group for use with an ELB or without.

See the module we have created here: https://github.com/unifio/terraform-aws-asg

We still need to implement each use case separately unfortunately, but leveraging the module from another stack works pretty much the way I would want driven by the value of data passed in and without the need for feature flags.

Using signum, we were able to simplify the logic required to create the toggle on and off logic for various resources.

resource "aws_autoscaling_group" "asg" {
  count  = "${signum(length(var.min_elb_capacity)) + 1 % 2}"
.
.

resource "aws_autoscaling_group" "asg_elb" {
  count  = "${signum(length(var.min_elb_capacity))}"
.
.

We are then using coalesce to return the ID of the resource that happens to be generated and returning it as a generic output to the calling template.

output "asg_id" {
  value = "${coalesce(join(",",aws_autoscaling_group.asg.*.id),join(",",aws_autoscaling_group.asg_elb.*.id))}"
}

[oillio]

If you add support for conditional logic on resources, please add it to fields within resources as well.

I would like to be able to do something like this:

resource "aws_elastic_beanstalk_environment" "myEnv" {
  name = "test_environment"
  application = "testing"

  setting {
    onlyif = var.min_nodes != ""
    namespace = "aws:autoscaling:asg"
    name = "MinSize"
    value = "${var.min_nodes}"
  }
}

[ketzacoatl]

@oillio , I recently learned about coalesce() - https://www.terraform.io/docs/configuration/interpolation.html#coalesce_string1_string2_ - and it looks like you might also benefit from that function, I think it would meet your intention/need in that example snippet.

[oillio]

How could coalesce be used in this case? I want to avoid setting the MinSize beanstalk value under a given condition...

[ketzacoatl]

@oillio, well, I would think you could say "use this variable or the default", with null being the default, but I can see how you might want that to be more explicit, or exclude the setting completely if the variable is not set. I guess that would in part depend on the AWS API.

[oillio]

For this exact example, it would be even easier to just set a default value for the min_nodes variable.

MinSize is not the best example. There are a number of beanstalk fields that don't lend well to default values. For instance, environment variables, imageId, etc. Additionally, a lot of default values clutters the terraform output result.

[blakeneyops]

The only way coalesce helps you out here is with very specific conditions. It would not be as flexible as you would like it for arbitrary settings.

An example that would work though is as follows:

variable "min_nodes" {
  default = ""
}

resource "aws_elastic_beanstalk_environment" "myEnv" {
  count = "${signum(length(var.min_nodes)) + 1 % 2}"
  name = "test_environment"
  application = "testing"
}

resource "aws_elastic_beanstalk_environment" "myEnv_with_min" {
  count = "${signum(length(var.min_nodes))}"
  name = "test_environment"
  application = "testing"

  setting {
    namespace = "aws:autoscaling:asg"
    name = "MinSize"
    value = "${var.min_nodes}"
  }
}

output "myEnv" {
  value = "${coalesce(join(",",aws_elastic_beanstalk_environment.myEnv.*.id),join(",",aws_elastic_beanstalk_environment.myEnv_with_min.*.id))}"
}

The resource that ends up being created, determined by whether min_nodes is set to anything other than an empty string, would be returned as the output.

Again, this would not be a great fit to model numerous combinations, but if there is one or two specific conditions you are interested in, this will work.

[fwisehc]

How do I prevent this error? I'm trying to use count to disable creating a subnet and I have an output variable that I want to use for when the subnet is created.

* Resource 'aws_subnet.api_public_z3' does not have attribute 'id' for variable 'aws_subnet.api_public_z3.id'

Where:

variable "zone3_enabled" {
    default = "0"
}
output "subnet_data_sql_z3" {
    value = "${join("", aws_subnet.data_sql_z3.*.id)}"
}
resource "aws_subnet" "data_sql_z3" {
    count = "${var.zone3_enabled}"

    vpc_id = "${var.vpc_id}"
    cidr_block = "${var.cidr_prefix}.202.0/24"
    availability_zone = "${var.vpc_zone_3}"
}

[nbr]

@fwisehc It appears that the following could be happening: assuming count is {0,1}, when count = 0 invoking join("", aws_subnet.data_sql_z3.*.id) has a dependency on aws_subnet.data_sql_z3.0.id, which does not have an attribute id because it does not exist.

[lbernail]

We use the count=${length(compact(split(",", var.foo)))}trick a lot (I just discovered the ${signum(length(var.bar))} one) for conditional resource creation but a separate property like "when" mentioned above would make our tf files easier to read

In addition count does not apply to sub-resources: in several cases we would like to be able to create sub-resources conditionally (listeners on ELB are a good example)

I have used cloudformation a lot in the past (and really prefer terraform, no question about that) and the conditions in cloudformation were enough for all the use-cases I remember.

A quick example: Declare conditions

  "Conditions" : {
    "CreateBastion" : {"Fn::Equals" : [{"Ref" : "BuildBastion"}, "Yes" ]},
    "CreateAdminSubnet" : {"Fn::Not" : [ {"Fn::Equals" : [{"Ref" : "AdminSubnet"}, "" ]} ] }
  }

Use it for conditional creation of a resource:

  "BastionInstance" : {
      "Type" : "AWS::EC2::Instance",
      "Condition" : "CreateBastion",
      "Properties" : {...}
   } 

Use it for if / then / else constructs:

"SubnetId" : {"Fn::If" : [ "CreateAdminSubnet",{"Ref" : "AdminSubnet"},{"Ref" : "PublicSubnet"}]}

[pmithrandir]

Hello,

On my side, I don't need so much conditional in the terraform script, but in template, it's essential.

For example, I'm generating n instance for my application server, and an HA proxy right after. This haproxy needs to get all the instances inserted in a config file... but It's almost impossible to insert multiple line using terraform.

Example : from a list of instances, you want to generate these line :

    server SERVER1  10.100.0.10:1080 check inter 2s fall 3 rise 1 maxconn 10000
    server SERVER2  10.100.0.11:1080 check inter 2s fall 3 rise 1 maxconn 10000
    server SERVER3  10.100.0.15:1080 check inter 2s fall 3 rise 1 maxconn 10000

each of the server name and the IP being from an instance object of course.

It's driving me nuts, when actually, a simple foreach in the template would have done in in 5 sec.

My solution today : (be careful it's spicy... and the server number doesn't increment actually) In my terraform file

resource "template_file" "haproxy_config_file" {
  template = "${file("${path.module}/templates/provisioner/haproxy.cfg.tpl")}"
  vars {
    backend_server_lines = "${base64encode(join("ENDBEGIN", openstack_compute_instance_v2.myserver.*.network.0.fixed_ip_v4))}"
    instance-count = 0
  }
}

In my template file :

${replace(replace(concat(concat("BEGIN",base64decode(backend_server_lines)),"END"),"BEGIN",concat(concat("server SERVER",instance-count + 1)," ")),"END",":1080 check inter 2s fall 3 rise 1 maxconn 10000\n    ")}

I'm not sure I should be proud of it, but the line are almost generated... I miss only the server name to be unique... but I can't avoid to be ashamed of my code, I don't think I will be able to read it tomorrow anymore.

[mengesb]

@pmithrandir

The simpler solution would be a script to generate the file instead of your creative solution.

[pmithrandir]

@mengesb hi. A template engine for example...

Just kidding

[geekq]

Terraform allows only very very basic value interpolation in the template_file resource https://www.terraform.io/docs/configuration/interpolation.html

If instead it would allow full-blown golang templates like https://github.com/hashicorp/consul-template#templating-language, it would solve most conditional logic / iteration problems, e.g. one described by @pmithrandir

I have not found any terraform plugin, offering that feature. Lets see if I can write one myself...

[ketzacoatl]

Apologies if I am stating the obvious.. using golang templates might be nice for templates, scripts, etc.. but does not cover conditional resources.

[bfgoodrich]

Has there been any more though/progress on using "where/when" to define conditional logic for resources?

[grebois]

+1

[FastNinja]

would be nice to have conditionos

[ghost]

+1 for conditional evaluation. With tf being declarative, I tend to lean towards a single statement ternary approach like val = ${cond(eval_statement, entity_when_true, entity_when_false)}.

Also +1 for supporting golang template syntax.

[ryanjjung]

I have a situation where I'd like to use the Spotinst service (who provide a terraform plugin I use) to build all environments except production, where I require an ASG production dedicated instances. Because the resource types differ, it's complicated to build terraform in such a way that it builds this tier using different resource types in different environments.

[brikis98]

If you're looking for a way to do if-statements in the meantime, I wrote up a blog post that captures a few of the basic techniques that are possible today: Terraform tips & tricks: loops, if-statements, and gotchas. It's not as nice as having built-in language support for conditionals, but it's surprising just how far you can go with creative use of the count parameter.

[cmcconnell1]

Thanks @brikis98 for creating the above page. It has been very helpful for me with this issue and others.

[Joosakur]

Yep, the page by @brikis98 is very helpful, thank you. There are some cases which are not probably covered by these workaround hacks though.

For example, I'd like to include https listener in aws_elb only if the (in that case mandatory) certificate id variable is not empty/null. Since the listener parameter has a type of object array, the string interpolation functions don't seem to apply to it easily.

[jgartrel]

@phinze - My kingdom for a case statement:

  # Count = 1 only if 
  #   provider = aws AND nodes != 0 AND length(vpc_security_group_ids) is 0
  count = "${lookup(map("aws", "${signum(var.nodes)}"), "${var.provider}" , "0") * ( signum(length(var.vpc_security_group_ids)) - 1 ) * -1 }"

or a Ternary Operator and some logic operators

Fantastic thread, and thank you!

As the Oracle would say ... "Oh, don't worry about it. As soon as you step outside that door, you'll start feeling better. You'll remember you don't believe in any of this logic crap. You're in control of your own DSL, remember?

[ahammond]

I think the ongoing lesson is "Don't write your own DSL. It will suck. You'll have to write a parser and editor/IDE support and those will suck, too." Saltstack was mentioned before in the context of preprocessors, but I think the fundamental lesson there is actually that it's domain specific language isn't a DSL, it is just a data-structure. You can encode that data-structure pretty much however you want (jinja pre-processed yaml is the standard, but there are plenty of other options). The important thing is that it renders down to a data-structure which is de-serialized and then drives the functionality. This seems to be a design paradigm that doesn't suck.

[mitchellh]

@ahammond Which is the design paradigm of Terraform: we fully support JSON for this reason and projects out there do use full languages like Ruby to generate Terraform JSON. I'm not sure if you were saying Terraform didn't do this, but I want to make it clear that Terraform is a data structure currently and is not a DSL.

HCL can definitely be viewed as a DSL, but its really just a human-friendly language for writing data structures. We don't support conditionals or other non-data structure features yet so you can't consider it much more than that...

However, I've also learned the lesson (via Vagrant) of "don't use a full programming language" as well because people do really crazy things that make it difficult to safely load and use configuration.

[ahammond]

@mitchellh that's awesome! I'm going to take a look into the JSON format! I'm super-new to Terraform, found this issue while looking for a way to iterate that didn't suck.

[apparentlymart]

FYI to those who are following this issue: Terraform 0.8 will have some basic support for conditionals and boolean operations in the interpolation language, giving us some first-class support for conditionals, albeit only inside interpolations for now.

Although we don't yet have an explicit when or if meta-attribute as was discussed above, this new conditional operator makes it easier to achieve that using count:

    # Create this only in production
    count = "${var.environment == "production" ? 1 : 0}"

Seems likely that a hypothetical future meta-attribute for explicitly enabling/disabling resources conditionally would build on the new boolean operators; for now, the ternary conditional operator should hopefully tidy up some of the existing tricks people were doing with setting count conditionally.

[mtougeron]

@apparentlymart This is awesome, thanks!

[ketzacoatl]

Very excited to leverage this with TF 0.8.x, thank you for all the great work here!

[aavileli]

awaiting for TF .8

[ryno75]

IT'S HERE! YAHOOOOOOOOOOOOOOOOOOO!!!

[stack72]

Closing this as it has landed in Terraform 0.8.x :)

[tmatilai]

@stack72 I assume you refer to the new ternary operator? While that covers many of the use cases, it won't help with optionally specifying attributes, or "sub hashes" inside the resources. For example optionally declaring multiple listeners in an ELB. Would be nice to have an issue for tracking those, too.

But great work, the ternary operator for sure makes things easier especially in generic modules!

[apparentlymart]

Given the age and large potential scope of this issue, I guess it makes sense for us to close it out and capture some more-specific use-cases in other issues, since this one was originally motivated with just collecting information on patterns people were using, and didn't have a tight enough scope that it would ever likely be closed by any real implementation work.

The trick there, of course, is that we previously intentionally consolidated several issues describing other use-cases over here, so there's a bunch of closed issues linked from here that capture some real use-cases we presumably don't want to "lose".

FWIW though, I understand that improved conditional stuff is still on the radar even if this particular issue is closed.

[oillio]

@tmatilai - Would this feature request solve your issue? https://github.com/hashicorp/terraform/issues/7034

[tmatilai]

@oillio yeah, as far as I understand, that would cover the sub-resource/block case!

Even after that one use case would be conditional individual attributes, although I haven't personally been so much in need of those compared to sub-resources.