Question: Suggestions for hcl.Body manipulation

[floresj]

Hola! I've been working on some tooling that leverages hcl2 and it has been working beautifully. I am currently prototyping a few things, one of them being the ability to merge attributes from n bodies. However, I'm struggling to wrap my head around how to best accomplish this and wanted to get some tips. Say we have two configuration files (base.hcl, dev.hcl) with the following contents:

// base.hcl
env = {
   foo = "bar"
   bar = mycustomfunc("foo")
   foobar = var.foobar
}

and

// dev.hcl
env = {
   foo = "b-a-r"
   bar = "f-o-o"
}

In this simple scenario, I'd like to be able to perform a shallow merge of the key/values from the env attributes to generate one attribute containing the merged content. Further, I want to defer any EvalContext evaluation until I have generated the merged attribute.

For instance, I want to end result to be something along the lines of:

env = {
   foo = "b-a-r"
   bar = "f-o-o"
   foobar = var.foobar
}

I was trying to use hclwrite to parse a configuration and then manipulate it. However, I'm struggling to find a way to actually generate the expressions making up the env attribute.

I took a look at the include package under extensions to get some ideas but I'm not still not sure.

How should I go about achieving this? I feel like l'm thinking about this too much and missing something simple. Any tips or suggestions would be appreciated.

[apparentlymart]

Hi @floresj,

The hclwrite package is still evolving, so it's not yet sufficient to meet all generation use-case. The general approach for what you're talking about here would be to detach the attributes or expressions from your original hclwrite bodies and then attach them to the new body without modification, thus allowing hclwrite to preserve the expression as-is, but I'm not sure if all of the necessary API methods are there to do that right now.

This repository is currently in a sort of frozen state because we're working on bringing this code back into the proper hashicorp/hcl repository and making a stable release tag of it, so unfortunately I don't think we'll be able to add that new functionality right now, but hopefully we'll get HCL v2.0.0 tagged soon and then we can re-open the tree to new PRs over in the other repository.

---

In the meantime, depending on how general you need your solution to be you might be able to get away with some string-based templating instead of syntax tree manipulation: if you look at the expression object associated with each attribute you will see that it has a method Range.

If you know for certain that the input is HCL native syntax, you can use Range.SliceBytes to slice out the source bytes for that expression and then paste it into your output verbatim, generating the rest of the output with string concatenation instead. If you then pass the result to hclwrite.Format then should should get essentially the same result as the syntax-tree-based technique.

This is more-or-less how Terraform's terraform 0.12upgrade tool works, because it was written before hclwrite was usable even for basic config generation and also has to deal with input in HCL 1 syntax.

This won't work if you need to make surgical edits in the middle of a larger file, of course. That's what hclwrite will hopefully evolve to deal with better.

[floresj]

Awesome, thanks a bunch for the insight. I'm going to give the string based templating a shot.