Closed phinze closed 7 years ago
Here are 2 examples: https://gist.github.com/chrisferry/780140d709bfad51038c The RDS and ELB modules have minor differences. SSL cert or no. IOPs or no.
OMG, I could rant on about this issue for a long while. One litany of clear and concise use cases are found in implementing a concept as a terraform module. There are even community modules which exemplify this.. two modules for essentially the same thing, one provides an ELB, one does not.
I tend to want to write terraform source as I do with Saltstack: as a giant jinja template. This affords me a whole lot of flexibility while ensuring the application (Salt) ends up with a machine-readable format. Terraform sort of has this type of pre-processing (with interpolation), but Saltstack's implementation leverages the concept of a pluggable renderer system.. this pre-processor renders the template to give to salt for processing. The renderer can be jinja, mako, or any one of a few different systems (made available as modules). The user/developer experience has been exceptional, and I am thankful for the power it lends, while still providing a declarative system. In contrast, using terraform has felt cumbersome and restrictive in the expression of one's needs (especially when I have gone to encapsulate a working POC into a module).
Thank you for opening this dicussion!
This is a bit of a stretch on the topic of this issue, but a couple of times I've found myself wishing for an iteration construct to allow me to create a set of resources that each map one-to-one to an item in a list.
I've found and then promptly forgotten a number of examples (having dismissed them as impossible), but one that stayed in my mind was giving EC2 instances more memorable local hostnames and then creating Route53 records for each of them.
resource "aws_instance" "app_server" {
# (...)
count = 5
provisioner "remote-exec" {
inline = [
"set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
]
}
}
foreach "${aws_instance.app_server.*}" {
resource "aws_route53_record" "app_server-${item.private_ip}" {
zone_id = "${something_defined_elsewhere}"
name = "appname-${join(\"-\", split(\".\", self.private_ip))}.mydomain.com"
type = "A"
records = ["${item.private_ip}"]
}
}
In my imagination, this creates a set of resources named things like aws_route53_record.app_server-10.0.0.1
which then get created/destroyed as you'd expect when the corresponding instances come and go. In case it wasn't obvious, item
here is imagined as the iteration variable.
While I was sketching this out I also came up with an alternative formulation that might end up leading to a simpler mental model:
resource "aws_instance" "app_server" {
# (...)
count = 5
provisioner "remote-exec" {
inline = [
"set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
]
}
child_resource "aws_route53_record" "hostname" {
zone_id = "${something_defined_elsewhere}"
name = "appname-${join(\"-\", split(\".\", parent.private_ip))}.mydomain.com"
type = "A"
records = ["${parent.private_ip}"]
}
}
In this formulation, rather than generically supporting iteration over all lists we can just create a family of child resources for each "parent" resource. This feels conceptually similar to how per-instance provisioners work. I'm imagining that the child resource would interpolate like ${aws_instance.app_server.0.children.hostname.records}
, or indeed ${aws_instance.app_server.*.children.hostname.name}
to get all of the created hostnames.
When your environment always looks the same (e.g. for a long standing/running app), the declarative language of terraform is expressive enough for most needs. However, my use case (frequent spin ups/tear downs of AWS VPCs of a similar general structure but with plenty of instance/subnet variation) means that terraform is not the "start" of my pipeline. I need to combine some configuration, logic, and templates on the fly each time to define my desired environment before terraform can ingest it.
Right now, I'm planning on writing something custom (rake and erb?) to generate the needed terraform json and go from there because I don't see template-esque logic as a job for terraform's declarative config language, It already can use json as an incoming interchange format, so wouldn't it be more versatile to just leverage any of the existing template renderers out there as @ketzacoatl described? It would keep logic out of the configuration and keep the terraform language simple/clean.
Appreciate this discussion and I'm open to learning about a better way to solve these kinds of problems.
And I just realized why my suggested approach is flawed in some cases. The power of terraform is using derived data at runtime as variables elsewhere. If you render separately ahead of time in some cases, you lose that. The count=5
is an example of something you can't pre-render and then reference easily. Gah, sorry.
The power in terraform, IMHO, is that we have the flexibility to choose how much we do before , in TF, and after TF runs. In most cases, you need to start working with some wrapper to create the JSON you want, when the existing interpolation syntax won't get you what you want. I personally, have avoided this as I would rather keep the before limited to a CI / admin who defines details in the terraform.tfvars
file to pass to TF when it runs. At the same time, this is also where the conditional logic and other interpolation/syntacitic sugar comes in (but is in some cases missing).
Last December in AMS Dockercon I had asked @mitchellh if there would be any plans to add such logic control in Terraform DSL, he explained his view on keeping Terraform as simple as possible maybe adding a little algebraic functionality (which has already been merged) and standard string operations.
I am glad there are second thoughts on this, but it is a decision that needs a lot of input and real world justification, so thatnks @phinze for bringing this up.
I have two real world scenarios I 've faced where an if
statement would have come in handy in Terraform:
rs.add()
itself by connectiong to the master node.
Right now I 've resorted in a nasty script to get the job done along with a dummy resource to invoke a provisioner.
I am sure there must be a more elegant way, irregardless if
would greatly mitigate the nastiness in my setupif
would be nice to instruct a remote-exec
provisioner to work its magic on just one node. Right now I just create a dummy resource that depends on .0.id
so that it gets provisioned from there. Ugly. There are ways around it, but still...Other scenario,
A simple scenario -
Optionally use Atlas artifacts to deploy infrastructure - and fallback to just AMI strings (if not using atlas).
Something like -
if ${var.atlas_enabled} {
resource "atlas_artifact" "machine" {
name = "${var.atlas_artifact.name}"
type = "aws.ami"
}
}
then in a resource
resource "aws_instance" "machine" {
if ${var.atlas_enabled} {
ami = "${atlas_artifact.machine.id}"
} else {
ami = "${var.ami_string}"
}
}
:+1:
I like the idea of even a simple
if ${var.atlas_enabled} {
}
There are lots of times where I'm building the same infrastructure for dev/stage/prod but don't need things in dev/stage as are needed in prod.
Even simpler, doesn't break the current syntax and prevents complexity (Inspired by Ansible):
resource "aws_instance" "machine" {
when = "${var.atlas_enabled}"
ami = "${atlas_artifact.machine.id}"
}
resource "aws_instance" "machine" {
when = "not ${var.atlas_enabled}"
ami = "${var.ami_string}"
}
@franklinwise that's pretty nice!
I like that it preserves the declarative syntax.
Preserving the declarative syntax seems like priority number one in my opinion. I like the 'when' solution. :+1:
Yes, I too like this suggested syntax.. when
, and the use of not
here, makes a lot of sense.
This seems inadequate as it only supports the conditional creation of resources. In my usage, I've found myself wanting conditional expressions (not statements) several times.
The use case has been creating heterogenous groups of ec2 instances. I.e., I'd like to create 12 instances where the first 4 are of type m3.large and the remaining are c4.xlarge. I've hacked around this for the time being by using a lookup table and creating an entry for each index, but it's pretty nasty.
For clarification, what I'm looking for is something like:
instance_type = "${if count.index > 3 then c4.xlarge else m3.large}"
It might make sense to consider these use cases separately, if only for the goal of getting the simpler implemented faster than the more complicated and nuanced case. when
/ not
as a field on all resources is a fantastic start. Figuring out the rest seems to be too much to figure out in the immediate.. why block one for the other?
Expanding on the referenced aws spot instances - I'd like to be able to express "spin up N instances in at least M availability zones, bidding the current bid price * X, But fallback to on-demand for any az where the bid price is > Y, or spot instance is unavailable". That seems complex (I have implemented it via a custom ruby script now), so maybe it's less a "we need conditionals" argument and more a "it would be nice if the aws provider abstracted instance types and did the "right thing").
Maybe the whole conditional thing could be handled by some sort of "call to external" hook to work the logic? It's a cop-out, in a way, but perhaps the most flexible in the end. If the suggested "when" could call an arbitrary script, with current context, you could kinda wedge in anything you needed as a shim, in only the spots where declarative is problematic. Can terraform do external calls like that already? I came into this from the side googling for spot instance solutions and seeing if terraform had support, so I'm not super familiar with current functionality there.
:+1: Really like @franklinwise suggestion. I think this is the way to go if control statements are added.
Just to add another use case similar to rafikk's example. I'm using terraform to bring up a count of 'container instances' or 'nodes' in an AWS ECS cluster with consul running on each one. I want the first three to be run as servers but the rest to be agents. Currently I have to duplicate the resource block which isn't very neat and any changes have to be done twice which allows too much room for human error.
It would be more flexible if I could do the following:
user_data = "${if count.index < 4 then SERVER_SCRIPT else AGENT_SCRIPT}"
@RJSzynal We have the same thing in our configuration. I really think support conditional logic inside of interpolation blocks is a must.
tl;dr my five cents is that I'm for the high-level idea of better conditional / looping support, but would like to stick to keeping it declarative.
I like the child resource proposal by @apparentlymart (far more than the foreach, which feels too imperative). That being said, for that specific example I feel like the resource isn't really a child. What may be better is a top-level "group" construct, which supports count
-style looping.
I'm also +1 for some basic conditional structure that gets interpolated. I don't like the imperative style "if then else" that's been recommended. I'd rather stick with a more procedural style thing that we already have, e.g., user_data = "${cond(var.use_foo, "foo", "bar")}"
, or perhaps some basic conditions user_data = "${cond(var.count = 1, "foo", "bar")}"
I agree with @rafikk with the lookup table being too verbose:
variable "lookup_table" {
default = {
"true" = "foo"
"false" = "bar"
}
}
variable "use_foo" {}
resource "aws_instance" "my_instance" {
ami = "${lookup(var.lookup_table, var.use_foo)}"
...
}
I'm also not fond of the if block surrounding a resource, as things start looking too imperative. A better approach to stick with the declarative format would be something like:
resource "aws_instance" "my_instance" {
ignore = "${var.disabled}"
}
I think setting count = 0
essentially does this, so this may be redundant. Sorry if I've repeated any ideas that may have already been expressed in earlier comments!
:+1:
@thegedge sorry I just noticed your response to my earlier example even though you posted it a while back.
The special thing I was imagining for "child resources" is that they'd always have an implied dependency on their parent, so if you delete the instance then that always deletes the record along with it... perhaps "child" is the wrong word, but I was going for "this thing only exists to support the thing it's nested inside".
@apparentlymart Ah yes, that would be nice. Could that also be solved with a "also destroy dependent resources" flag to terraform destroy
? Maybe modeling these things as being intimately connected (i.e., an atomic unit) would be useful in other, not immediately obvious ways too.
Anyways, I'm also going to add in an example from my team, since I didn't do that in my last comment and that's what @phinze was interested in seeing! We want our app developers to build things on top of a terraformed "cloud", but we want them to be able to this with a minimal configuration that doesn't require much/any terraform knowledge.
Many of our apps follow a similar recipe: rails app that sometimes needs redis, memcache, an S3 bucket, a place to run the rails app, and/or a database, maybe some other things. We'd like to construct a module that allows our app developers to conditionally select what they need for their app. It would look something like this from the app developer's perspective:
## /my_app/main.tf
module "my_app" {
source = "../app_template"
rails_server = "puma"
background_jobs = "resque"
memcache = true
}
## /app_template/main.tf
...
resource "aws_elasticache_cluster" "memcache" {
ignore = "${!var.memcache}"
...
}
resource "aws_elasticache_cluster" "redis" {
ignore = "${var.background_jobs != "resque"}"
...
}
...
An example I just ran into was attempting to create a list of users on AWS:
variable "devops" {
default = "user1,user2,user3"
}
resource "aws_iam_group" "Administrators" {
name = "Administrators"
path = "/"
}
resource "aws_iam_user" "devops" {
count = "${length(split(",", var.devops))}"
name = "${element(split(",", var.devops), count.index)}"
}
This creates a reasonable create plan
+ aws_iam_user.users.0
arn: "" => "<computed>"
name: "" => "user1"
path: "" => "/"
unique_id: "" => "<computed>"
+ aws_iam_user.users.1
arn: "" => "<computed>"
name: "" => "user2"
path: "" => "/"
unique_id: "" => "<computed>"
+ aws_iam_user.users.2
arn: "" => "<computed>"
name: "" => "user3"
path: "" => "/"
unique_id: "" => "<computed>"
But now user2 leaves.
-/+ aws_iam_user.users.1
arn: "arn:aws:iam::909704556315:user/user2" => "<computed>"
name: "user2" => "user3" (forces new resource)
path: "/" => "/"
unique_id: "AIDAIOESVNFST3M3THMO6" => "<computed>"
And the list gets shuffled down, deleting a legitmate user along the way.
A loop and a way to use a variable in a resource identifier would resolve this.
If you are building a module for any resource like RDS or any resource that change behaviour when additional attributes have been set then a evaluating function is required like the following which is available in cloudformation
"MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "5", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : { "Ref" : "DBUser" }, "MasterUserPassword" : { "Ref" : "DBPassword" }, "DBParameterGroupName" : { "Ref" : "MyRDSParamGroup" }, "DBSnapshotIdentifier" : { "Fn::If" : [ "UseDBSnapshot", {"Ref" : "DBSnapshotName"}, {"Ref" : "AWS::NoValue"} ] } } }
AWS Elasticache Redis has a snapshot parameter. Sometimes we want to pass in a snapshot, and we set this param: snapshot_arns = ["${var.redis_snapshot}"]
Other times we do not want to pass in snapshot, however if you pass in an empty string, the elasticache cluster will not be built as its an invalid request. Even using modules and count=0, we cannot get around the conditional nature of this.
The use case that I ran into today was wishing for:
resource "aws_elb" "maybe-ssl-elb" {
listener {
instance_port = 80
instance_protocol = "http"
lb_port = 80
lb_protocol = "http"
}
if (not empty("${var.elb_server_certificate_arn}")) {
listener {
instance_port = 443
instance_protocol = "https"
lb_port = 443
lb_protocol = "https"
ssl_certificate_id = "${var.elb_server_certificate_arn}"
}
}
}
I have a module which I was using for each environment "./app" now I've split it up into "./app-prod" "./app-stage" etc because some environments use user_data
some do not. And the ones that do not I can't pass in a variable that says null
. An if
statement would allow us to do something like:
if "${var.userdata}" {
user_data = "${var.userdata}"
}
Of course having a null
or something would be good too. I might be missing something obvious.
I wanted to at least drop in an interesting way of doing if ! x.empty y else empty
right now, which we use to choose whether or not we chef provision with a private or public IP:
host = "${replace(self.private_ip, replace(var.bastion_hosts, "/^$/", "/^.*$/"), "")}"
If var.bastion_hosts
is empty, we end up replacing all of self.private_ip
with an empty string. For the connection host
parameter, this means terraform will pick the value for us. If it's non-empty, we search for var.bastion_hosts
, which should never occur in self.private_ip
, so we get the value of self.private_ip
back.
UPDATE:
My assumption that an empty string would have terraform pick the host was wrong, so we had to go full ternary. Here's how you do if foo empty then bar else baz
. If foo is a substring of baz this doesn't work.
${replace(replace(var.baz, replace(var.foo, "/^$/", "/^.*$/"), ""), "/^$/", var.bar)}
nice pattern, thank you for sharing! I'll look forward to the day when this is not necessary, but I imagine it'll do for now (the simple use case, that is).
@thegedge thanks for sharing, neat hack
Some use-cases for looping (along with a proposal for supporting them) in #3310.
+1.
Right now I'm working around this by applying tf files in one folder, or another folder conditionally.
Conditionals would be useful in our dns module, which wraps route53 or ultradns depending on the application lifecycle environment. To work around this, we're writing a terribly ugly workaround:
variable "dns_provider" { default = "none" }
# supporting multiple dns providers requires non-obvious code. It works
# by generating the count attribute from var.dns_provider. If the provider is
# supposed to be used, it will just pass count = 1 through. Otherwise, it
# defaults to count = 0.
# also note: because re2 doesn't support negative lookahead, we can't
# ensure that unexpected values are properly defaulting to count = 0.
resource "aws_route53_record" "record" {
count = "${replace(replace(var.dns_provider, \"route53\", 1), \"/\A(ultradns)|(none)\z/\", 0)}"
# other attrs here...
}
resource "ultradns_record" "record" {
count = "${replace(replace(var.dns_provider, \"ultradns\", 1), \"/\A(route53)|(none)\z/\", 0)}"
# other attrs here...
}
Hi @apparentlymart ,
Please I would like to know if this code block you wrote is what you expect or what is really support in terraform :
resource "aws_instance" "app_server" {
# (...)
count = 5
provisioner "remote-exec" {
inline = [
"set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
]
}
child_resource "aws_route53_record" "hostname" {
zone_id = "${something_defined_elsewhere}"
name = "appname-${join(\"-\", split(\".\", parent.private_ip))}.mydomain.com"
type = "A"
records = ["${parent.private_ip}"]
}
}
I didn't know the child_resource configuration before, nor the python-like syntax ...{join(\"-\", split(\".\", self.private_ip)...
And I don't see them in the terraform documentation. The inline argument allows bash commands and scripts but not python syntax for me.
I would to be clarified about that.
Thanks.
My bad, sorry,
I see join fucntion in the interpolation section.
@papiveron this issue is discussing possible new Terraform features; the constructs I included in that example were hypothetical, and not actually supported in Terraform today. Sorry for the confusion.
Seems like several people like my idea about using "when" @phinze @rossedman @knuckolls @ketzacoatl . Any expression that terraform supports could go in the when statement. If one wants to support looping then another declarative field of "with_items" could be used.
Here's an example of "with_items" (again, stolen from Ansible), where 'item' is a special keyword like 'var':
resource "aws_instance" "machine" {
ami = "${item.id}"
with_items = "${var.list_of_amis}"
}
and can be combined with "when"
resource "aws_instance" "machine" {
ami = "${item.id}"
when = "${var.custom_ami | length > 0}"
with_items = "${var.list_of_amis}"
}
One use case for me is the ability to use the same code to construct VPCs, but include or not include certain modules based on the value of variables. For example, a simple if... clause (I don't even need the 'then' portion)
if $include_directConnect { module "direct_connect" { source = "git::ssh://git@stash:7999/tf_module/direct_connect" ... } }
This would allow me to use the same VPC code across different development groups and different environments (dev, qa, stage, prod) without having to remember to comment/un-comment the code every time. It would be a simple matter of setting the variable in that group/environment config file and then just running get & apply.
Very similar to earlier mentioned cases. I'm running into this in module use. I'm creating several VPCs in my AWS cloud. Some with public connectivity, some without. Writing multiple modules to handle that feels very wasteful. Some kind of flow control would make this much cleaner.
Ah, having played some more, I think the count example is insufficient.
Say, I want to do something between 0 and N times. The common way this seems to get done, is via something like count = "${length(split(",", var.private_subnets))}"
However, this results in either needing trailing commas, or being unable to handle the 0 case:
Eg:
${length(split(",", ""))}
is 1${length(split(",", "foo"))}
is 1${length(split(",", "foo,bar"))}
is 2${length(split(",", "")) -1 }
is 0${length(split(",", "foo")) - 1}
is 0${length(split(",", "foo,")) - 1}
is 1${length(split(",", "foo,bar")) - 1}
is 1neither of those is very satisfying
@directionless FYI, the compact
function was somewhat recently added to deal with empty strings when splitting so, for example:
${length(compact(split(",", "")))}
should be 0${length(compact(split(",", "foo")))}
should be 1${length(compact(split(",", "foo,bar")))}
should be 2@thegedge Indeed! I'd just found that in the docs, and testing, that does seem to work.
I'll PR to the modules I need it in.
I'm using count to conditionally execute a resource:
resource "cloudstack_secondary_ipaddress" "clustervip" {
count= "${var.servers.instance_count * length(split(",",var.requiredzones)) * length(split(",",var.servers.hostname)) * var.alwayson}"
virtual_machine = "${element(cloudstack_instance.servers.*.id, count.index)}"
}
Where var.alwayson is just a variable with 0 or 1.
@cihatgenc - What happens if you don't have any cloudstack_instance.servers ? Doesn't this blow up? I'd like to use similar binary logic to determine whether or not to create an aws_vpn_gateway, but can't figure out how that would work. I don't think it could.
@pll - in the example resource above I'm adding an ip to a cloudstack instance. So yes, if there is no server it will fail. I did not use the resource aws_vpn_gateway myself, but it could work if you add a count to the resource aws_vpc
Just to express the idea (don't know if this actually works):
variable "vpc_count" {
default = 1
}
variable "dovpn" {
description = "0 = No, 1 = Yes"
default = 0
}
resource "aws_vpc" "main" {
count = "${var.vpc_count}
cidr_block = "${var.vpc_cidr_block}"
tags {
Name = "${var.vpc_name}"
}
}
resource "aws_vpn_gateway" "vpn_gw" {
count = ${var.vpc_count * var.dovpn}
vpc_id = "${element(aws_vpc.main.*.id, count.index)}"
}
@cihatgenc - Interesting. So, if I create the VPC with a count variable, and I only create 1, then I have a splat list where aws_vpc.main.0.id is the ID of the first VPC in a 1-element list of VPCs? So, if I do not want a VPN gateway attached, I set the count of that to effectively to 0 ( vpc_count =1 * gw_count = 0), then index into the list of VPCs. But doesn't that result in assigning the VPN gateway to the 0th element of my 1-element list?
Sorry, I guess I don't really understand how splats work in this case.. thanks for your help and patience though :)
@pll - No, it will not assign it to the 0th element. When count = 0, TF will interpret it as "I have to do this resource 0 times", so will do nothing. When the count = 1, TF will do the resource 1 time on the vpc_id in the 0th element of the list (hence the count.index).
It's been important from the beginning that Terraform's configuration language is declarative, which has meant that the core team has intentionally avoided adding flow-control statements like conditionals and loops to the language.
But in the real world, there are still plenty of perfectly reasonable scenarios that are difficult to express in the current version of Terraform without copious amounts of duplication because of the lack of conditionals. We'd like Terraform to support these use cases one way or another.
I'm opening this issue to collect some real-world example where, as a config author, it seems like an
if
statement would really make things easier.Using these examples, we'll play around with different ideas to improve the tools Terraform provides to the config author in these scenarios.
So please feel free to chime in with some specific examples - ideally with with blocks of Terraform configuration included. If you've got ideas for syntax or config language features that could form a solution, those are welcome here too.
(No need to respond with just "+1" / :+1: on this thread, since it's an issue we're already aware is important.)