Unable to find json equivalent to hcl

[joernheissler]

I'm trying to autogenerate some terraform configs in json format. In HCL it works as expected, but with JSON I get errors.

HCL Template

locals {
    a = [ 
        {
            "b" = "c"
        },
        {
            "d" = "e"
        }
    ]
}

Same as JSON

{
  "locals": [
    {
      "a": [
        {
          "b": "c"
        },
        {
          "d": "e"
        }
      ]
    }
  ]
}

Expected behavior

Should work.

Actual behavior

* a: duplicate local. local value names must be unique

Steps to reproduce

1. run HCL input through json2hcl -reverse
2. save json output as foo.tf.json
3. run terraform apply

[VladRassokhin]

Probable it's because you have both .tf and .json files in the same directory. Terraform reads both and reports misconfiguration.

[joernheissler]

That's not the case. Did you try it yourself?

[VladRassokhin]

Ok, reproduced. Then isn't is a Terraform bug? Looks like there's no support for arrays for locals defined in json, since next snippet works fine

{
  "locals": [
    {
      "a": [
        {
          "b": "c"
        }
      ]
    }
  ]
}

[VladRassokhin]

Actually proper json for your case should be:

{
  "locals": [
    {
      "a": [
        [
          {
            "b": "c"
          },
          {
            "d": "e"
          }
        ]
      ]
    }
  ]
}

Unable to find json equivalent to hcl

Took just two minutes to find out that.

[joernheissler]

@VladRassokhin This one doesn't work either. I've got two files:

outputs.tf:

output "a" {
    value = "${local.a}"
}

locals.tf.json:

{
    "locals": [
        {
            "a": [
                [
                    {
                        "b": "c"
                    },
                    {
                        "d": "e"
                    }
                ]
            ]
        }
    ]
}

Error message from terraform apply:

Error: output.a: local.a: no local value of this name has been declared

I don't think that it's a terraform bug. Either it's a bug in the hcl library, i.e. what I want is not supported, despite the documentation saying otherwise "JSON can be used as completely valid input to a system expecting HCL". Or if there is a way to write it in JSON, then it's a problem with the documentation: It should provide better examples. And a formal specification of the HCL along with equivalents in JSON would be nice too.

[apparentlymart]

Hi all,

I think this is Terraform's bug rather than HCL's bug. HCL's dual-syntax approach relies on the calling application to properly interpret the decoded abstract syntax tree (AST), and usually that is done with Decode or DecodeObject, but Terraform has more specialized needs and so it works directly with the HCL AST objects. The exact AST structure can differ slightly between HCL and JSON, and so an application like Terraform that bypasses the decoder must be careful to handle those differences. The statement in the HCL readme applies only to HCL's own decoder, which Terraform does not use.

In particular, the locals block in Terraform is unusual in that it doesn't have a prescribed set of keys but rather Terraform walks the child nodes to see what keys seem to be present, treating each one as a definition. My guess is that the extra level of array in your JSON example is preventing Terraform from finding those keys when it walks. The usual way to write locals in JSON (and what we have in Terraform's test suite) is a single flat JSON object:

{
  "locals": {
    "a": [
      {
        "b": "c"
      },
      {
        "d": "e"
      }
    ]
  }
}

---

With all of this said, Terraform's more advanced use of HCL's low-level API is known to have a number of problems, which is why we're currently in the process of developing and switching to a new version of HCL whose API is more suited to the needs of an application like Terraform that needs to do gradual, custom decoding of the configuration data structure.

This new implementation has a more robust mapping to JSON which has a specification which interprets the JSON in a less-flexible way than the current implementation in return for avoiding confusing ambiguities such as what's described here.

Once Terraform is using that new implementation, a particular JSON structure will either be parsed as expected or it will produce an error explaining how to fix it. Since Terraform will no longer be using the implementation of HCL that's in this repository, the Terraform-specific problems that arise from it skipping Decode and DecodeObject will no longer be a concern.

[joernheissler]

:thumbsup:

Thanks for the explanation! I'll look forward to hcl2.