Rich template language

[errordeveloper]

As #1778 has provided a way of doing interpolation in files, next step would be to see whether there could be a richer language available from inside a template file.

Such language would help for generating provisioning data that could be passed to a provide as metadata for setting up VMs or basic configuration files to be uploaded to VMs.

[errordeveloper]

In one instance I have implemented a shell script that looks rather quite ugly for what it's able to do. It is being called from infra/main.tf#L44 and main.tf#L126.

[errordeveloper]

I should also clarify that in my use-case it's CoreOS that is being provisioned, and many other folks would probably suggest that a configuration management tool is what I want. However, I believe that this this kind of mechanism would be of use when one needs to seed the initial configuration files for their configuration management tool to run.

[apparentlymart]

It could be nice to use the same template language as consul-template, but with access to Terraform resources and variables within the calling module rather than access to Consul concepts like services and keys.

This could then allow Terraform provisioning to do the same sorts of things consul-template does, but for configurations that only vary per-deploy, rather than those that can vary at runtime. Having the template be able to access the resources directly means being able to easily iterate over list attributes and resource wildcards, which wouldn't be so easily to achieve with the explicit variables map in the existing template implementation.

Example haproxy template fragment, based on the one for consul-template:

listen http-in
    bind *:{{var "load_balancer_listen_port"}}{{range resource "aws_instance.server.*"}}
    server {{.id}} {{.private_ip}}:{{var "server_instance_listen_port"}}{{end}}

Of course, to achieve this would require a rather unusual resource type whose dependencies are defined by the contents of the external template file, rather than by interpolations within the resource definition itself.

[mitchellh]

I think we should just extend the template_file resource to support a type parameter that specifies the template type and we can support a variety there.

[errordeveloper]

Yes, :+1:, as long as there is a way of saying that "here is a file that needs to be processed in some way before it can be used". May be the types will have to declared somewhere, so user sees "You need tool foo to run this", instead of "foo: command not found" or something totally weird.

[progrium]

template_file with a way to provide types would be nice, but ideally if shell were one of them. Or perhaps support https://github.com/gliderlabs/sigil directly? We could probably do either or both as a PR depending on what sounds right.

[7heo]

@mitchellh :+1: that would be really great.

[ketzacoatl]

Saltstack makes the template/rendering engine plugable, so users can pick which renderer to use - it has worked out very well in practice and I believe that would be a great addition to Terraform's template provider!

[lbernail]

A richer templating engine would definitetly help! :+1:

[errordeveloper]

We had been using Jsonnet in Kubernetes Anywhere project, and it provides quite a few extra features for generating Terraform JSON then HCL does, it also have richer tooling for inlining plain text files, escaping JSON and rendering templates etc.

If you create a file named hello.txt with the following content:

Hello, my name is %(name)s, I am %(age)d years old.

you can import it from hello.jsonnet using importstr and substitute age and name variables from a dictionary to create an object with a meaningful message:

{
  message: (importstr "hello.txt") % {age: 25, name: "Rose"}
}

Run jsonnet hello.jsonnet and you will get:

> jsonnet test.jsonnet
{
   "message": "Hello, my name is Rose, I am 25 years old.\n"
}

There're other nice methods like std.escapeStringDollars and std.escapeStringBash.

[errordeveloper]

Please note, that with Jsonnet you would still make some use of Terraform interpolation, but only in case where you have to enforce dependencies. I think this is something worse documenting a bit more. Unlike some other kind of methods one could find for generating Terraform code, Jsonnet still keeps it declarative.

[flypenguin]

I'd also go for @apparentlymart's suggestion to use the consul templating languate. Make Hashi products consistent, and while it might not be the most intuitive one, it's "part of the family", already out there and being used.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.