virtuald / pyhcl

HCL is a configuration language. pyhcl is a python parser for it.
Mozilla Public License 2.0
336 stars 60 forks source link

Multiple keys with the same name #1

Closed maljac closed 8 years ago

maljac commented 9 years ago

According to: https://github.com/hashicorp/consul-template, HCL supports constructs like the following:

template {
  source = "/path/on/disk/to/template"
  destination = "/path/on/disk/where/template/will/render"
  command = "optional command to run when the template is updated"
}

template {
  // Multiple definitions are supported
}

When I parse such a config file using pyhcl (0.18), only the first template key is available:

{u'consul': u'someurl:8500', u'template': {u'source': u'/tmp/first.conf', u'destination': u'/tmp/config/first.conf', u'command': u'curl localhost || true'}}

The right way would probably to make the key template a list within the python dict:

{ ..., 'template': [{ _my_first_template_}, {_my_second_template_}], .... }
virtuald commented 9 years ago

Thanks for the bug report, I'll look into it tomorrow.

maljac commented 9 years ago

Any progress on this issue?

virtuald commented 9 years ago

Sorry about that, it totally slipped my mind. I'll look at it now.

virtuald commented 9 years ago

Confirmed bug, used hcl2json to generate expected json, will add a test/fix for this.

virtuald commented 9 years ago

Ok, I'm going to step back from this until https://github.com/hashicorp/terraform/issues/807 and https://github.com/hashicorp/hcl/pull/24 are resolved. HCL appears to be conflicted internally about what the output should actually be.

solarce commented 9 years ago

Just stumbled across this when discussing pyhcl with @phrawzty and we realized that https://github.com/mozilla/socorro-infra/blob/master/terraform/main.tf can't be easily converted to yaml because it's got a large number of resource definitions for the same kinds of resources, and they end up collapsed.

provider:
  aws:
    access_key: ${var.access_key}
    region: ${var.region}
    secret_key: ${var.secret_key}
resource:
  aws_elb:
    elb_for_webheads:
      availability_zones:
      - ${aws_instance.webheads.*.availability_zone}
      instances:
      - ${aws_instance.webheads.*.id}
      listener:
        instance_port: 80
        instance_protocol: http
        lb_port: 80
        lb_protocol: http
      name: elb-for-webheads
      security_groups:
      - ${aws_security_group.internet_to_elb__http.id}
  aws_instance:
    admin_host:
      ami: ${lookup(var.base_ami, var.region)}
      count: 1
      instance_type: t2.micro
      key_name: ${lookup(var.ssh_key_name, var.region)}
      security_groups:
      - ${aws_security_group.internet_to_any__ssh.name}
      - ${aws_security_group.private_to_private__any.name}
  aws_security_group:
    internet_to_snowflakes__http:
      description: Allow HTTP access to some oddball nodes.
      ingress:
        cidr_blocks:
        - 0.0.0.0/0
        from_port: 80
        protocol: tcp
        to_port: 80
      name: internet_to_snowflakes__http

https://github.com/hashicorp/terraform/issues/807 was closed with a fix in terraform and https://github.com/hashicorp/hcl/pull/24 seems to be awaiting code review, but it seems pretty clear that this is going to be a common way that HCL files are written for Terraform.

Is there something that can be done to handle this in the mean time? Knowing that pyhcl can handle Terraform configuration files would greatly help me as I could really use it in some Ansible modules I'm working on to help drive Terraform based deploys.

virtuald commented 9 years ago

I have zero bandwidth to work on this until after February 18.

Feel free to submit a PR with additional fixtures -- I think more examples need to be created that show this issue and their equivalent in json, so that we can be sure that both hcl and pyhcl are on the same page.

gposton commented 8 years ago

Here's another example of this issue.

There are 2 resources in this template: https://gist.github.com/gposton/27d5f63d2317eb5f75cb

But when parsed with pyhcl, the json only includes 1 resource.

Here's the output: https://gist.github.com/gposton/3fb0055348b4d8b79147

virtuald commented 8 years ago

I'm not motivated to fix this at the moment, but I'm happy to accept any PRs that will resolve this issue.

gposton commented 8 years ago

PR for this here: https://github.com/virtuald/pyhcl/pull/6

Excited to see this get in!

virtuald commented 8 years ago

Fixed in #6, pushed to pypi.