search

ansible-collections / cloud.terraform

The collection automates the management and provisioning of infrastructure as code (IaC) using the Terraform CLI tool within Ansible playbooks and Execution Environment runtimes.
GNU General Public License v3.0
98 stars 34 forks source link

bug: Variable insertions are malformed when executed with inline Terraform Variables #27

Open sean-freeman opened 1 year ago

sean-freeman commented 1 year ago
SUMMARY

Variable insertions are malformed when excuted with inline Terraform Variables.

Variable insertion into AWS Availability Zones does not work, due to Ansible Module for Terraform performing incorrect parsing.....

azs: '["{{ ansible_var_aws_region }}a", "{{ ansible_var_aws_region }}b", "{{ ansible_var_aws_region }}c"]'

results in -var 'azs=['\"'\"'us-west-2a'\"'\"', '\"'\"'us-west-2b'\"'\"', '\"'\"'us-west-2c'\"'\"']'

instead of -var 'azs=[\"us-west-2a\", \"us-west-2b\", \"us-west-2c\"]'

NOTE: related comment in Issue 24 copied below

*N.B. It is almost always better to preserve Terraform Input Variable formatting, to create a .tfvars and use operator -var-file=. For example, any input that is a SSH Key string will be mangled by -var=X and therefore instead use terraform apply -var-file=default.tfvars with key=value contained within.**

ISSUE TYPE
ANSIBLE VERSION
[:~] $ ansible --version
ansible [core 2.11.12] 
  python version = 3.7.8 (default, Oct 25 2020, 11:34:54) [Clang 12.0.0 (clang-1200.0.32.21)]
  jinja version = 3.1.2
  libyaml = True
COLLECTION VERSION
1.0.0 from Ansible Galaxy
1.0.1 from GitHub Repo main branch
OS / ENVIRONMENT
macOS 13.1
STEPS TO REPRODUCE
---

- name: "Ansible Playbook"
  hosts: localhost
  gather_facts: false

  vars_prompt:
    - name: ansible_var_aws_region
      prompt: Please enter AWS Region
      private: no

  tasks:

  - name: Terraform Module for AWS VPC - Terraform init and Terraform apply
    register: terraform_result
    environment:
      AWS_ACCESS_KEY_ID: "{{ ansible_var_aws_access_key }}"
      AWS_SECRET_ACCESS_KEY: "{{ ansible_var_aws_secret_access_key }}"
      AWS_REGION: "{{ ansible_var_aws_region }}"
    cloud.terraform.terraform:
      project_path: "{{ playbook_dir }}/tmp/terraform-aws-vpc"
      state: present
      force_init: true
      variables:
        name: "{{ ansible_var_aws_resources_prefix }}"
        ....
        azs: '["{{ ansible_var_aws_region }}a", "{{ ansible_var_aws_region }}b", "{{ ansible_var_aws_region }}c"]'
#        azs: '["us-west-2a", "us-west-2b", "us-west-2c"]'
EXPECTED RESULTS
ACTUAL RESULTS
fatal: [localhost]: FAILED! =>
{"changed": false, "msg": "Terraform plan could not be created
STDOUT: 
STDERR: 
Error: Invalid character

  on <value for var.azs> line 1:
  (source code not available)

This character is not used within the language.

Error: Invalid expression

  on <value for var.azs> line 1:
  (source code not available)

Expected the start of an expression, but found an invalid expression token.

COMMAND: /usr/local/bin/terraform plan -lock=true -input=false -no-color -detailed-exitcode
-out /var/folders/f5/xv_7b89d7ss448jmc1crwwl40000gn/T/tmp6ksbm5z3.tfplan
-var cidr=10.0.0.0/16
-var 'azs=['\"'\"'us-west-2a'\"'\"', '\"'\"'us-west-2b'\"'\"', '\"'\"'us-west-2c'\"'\"']'
...
anazobec commented 1 year ago

Hi, to fix your issue, you must do:

  1. use complex_vars: true before the variables block
  2. instead of writing the azs list as '["str_1", "str_2", "str_3"]', you should use: ["str_1", "str_2", "str_3"]

See the below example (based on your provided example) of this fix:

---
- name: "Ansible Playbook"
  hosts: localhost
  gather_facts: false

  vars_prompt:
    - name: ansible_var_aws_region
      prompt: Please enter AWS Region
      private: no

  tasks:

  - name: Terraform Module for AWS VPC - Terraform init and Terraform apply
    register: terraform_result
    environment:
      AWS_ACCESS_KEY_ID: "{{ ansible_var_aws_access_key }}"
      AWS_SECRET_ACCESS_KEY: "{{ ansible_var_aws_secret_access_key }}"
      AWS_REGION: "{{ ansible_var_aws_region }}"
    cloud.terraform.terraform:
      project_path: "{{ playbook_dir }}/tmp/terraform-aws-vpc"
      state: present
      force_init: true
      complex_vars: true  # <-- 1. fix
      variables:
        name: "{{ ansible_var_aws_resources_prefix }}"
        ....
        azs: ["{{ ansible_var_aws_region }}a", "{{ ansible_var_aws_region }}b", "{{ ansible_var_aws_region }}c"]  # <-- 2. fix
sean-freeman commented 1 year ago

Design reasoning for requiring complex_vars given in GH Issue #24

Resolved and closing accordingly

sean-freeman commented 1 year ago

Additional evidence even when using complex_vars: true and creating malformed variables

If executing an Ansible Task for a Terraform Module that exports a list, the list is malformed when entered into the next Ansible Task for a Terraform Module.

Example Terraform Module code

resource "aws_route53_zone" "dns_services_zone" {
  name = "rootdomain.tld"
  vpc {
    vpc_id = "vpc-abcdefgh"
  }
}

output "output_dns_nameserver_list" {
  value = aws_route53_zone.dns_services_zone.name_servers
}

Example Ansible code

# Example from integration tests in repo

- name: Set facts for all hosts
  ansible.builtin.set_fact:
    list_of_strings:
      - "kosala"
      - 'cli specials"&$%@#*!(){}[]:"" \\'
      - "xxx"
      - "zzz"

- ansible.builtin.debug:
    msg: "{{list_of_strings}}"

# Show output of previous Terraform Module
- ansible.builtin.debug:
    msg: "{{terraform_module1_result.outputs.output_dns_nameserver_list.value}}"

# Execute Terraform Module with the output of previous Terraform Module
- name: Terraform Module execution
  register: terraform_module2_result
  environment:
    AWS_ACCESS_KEY_ID: "{{aws_access_key}}"
    AWS_SECRET_ACCESS_KEY: "{{aws_secret_access_key}}"
    AWS_REGION: "{{aws_region}}"
    TF_LOG: "DEBUG"
    TF_LOG_PATH: "/tmp/terraform_debug.txt"
  cloud.terraform.terraform:
    project_path: "{{ playbook_dir }}/tmp/terraform_module_here"
    state: present
    force_init: true
    complex_vars: true
    variables:
      ...
      dns_nameserver_list: '{{terraform_module1_result.outputs.output_dns_nameserver_list.value}}'
      ...

- ansible.builtin.debug:
    var: terraform_module2_result

Ansible console output


TASK [Set facts for all hosts] *************************
ok: [localhost]

TASK [ansible.builtin.debug] **************************
ok: [localhost] => {
    "msg": [
        "kosala",
        "cli specials\"&$%@#*!(){}[]:\"\" \\\\",
        "xxx",
        "zzz"
    ]
}

TASK [ansible.builtin.debug] **************************
ok: [localhost] => {
    "msg": [
        "ns-0.awsdns-00.com.",
        "ns-1024.awsdns-00.org.",
        "ns-1536.awsdns-00.co.uk.",
        "ns-512.awsdns-00.net."
    ]
}

TASK [Terraform Module execution 2] **************************
failed: [localhost] => 
{
    "changed": false,
    "cmd": "/usr/local/bin/terraform apply -no-color -input=false -auto-approve -lock=true /var/folders/f5/xv_7b89d7ss448jmc1crwwl40000gn/T/tmp2m1d8s57.tfplan",
    "rc": 1,
    "stderr_lines": [
        "",
        "Error: Invalid index",
        "",
        "  on build_dns_update.tf line 39, in resource \"null_resource\" \"dns_resolv_files\":",
        "  39: nameserver ${var.dns_nameserver_list[0]}",
        "    ├────────────────",
        "    │ var.dns_nameserver_list is \"[\\\"ns-0.awsdns-00.com.\\\",\\\"ns-1024.awsdns-00.org.\\\",\\\"ns-1536.awsdns-00.co.uk.\\\",\\\"ns-512.awsdns-00.net.\\\"]\"",
        "",
        "This value does not have any indices."
    ]
}

DEBUG Terraform file

2023-01-06T14:53:22.845Z [INFO]  CLI args: []string{
"/usr/local/bin/terraform",
"plan",
"-lock=true",
"-input=false",
"-no-color",
"-detailed-exitcode",
"-out",
"/var/folders/f5/xv_7b89d7ss448jmc1crwwl40000gn/T/tmp2m1d8s57.tfplan",
....
"-var",
"dns_nameserver_list=[\"ns-0.awsdns-00.com.\",\"ns-1024.awsdns-00.org.\",\"ns-1536.awsdns-00.co.uk.\",\"ns-512.awsdns-00.net.\"]",
....
}

Examples from Hashicorp

Examples from Hashicorp always show Terraform Input Variables which are list or object types, using enclosing '.

terraform apply -var='image_id_list=["ami-abc123","ami-def456"]' -var="instance_type=t2.micro"
terraform apply -var='image_id_map={"us-east-1":"ami-abc123","us-east-2":"ami-def456"}'

export TF_VAR_availability_zone_names='["us-west-1b","us-west-1d"]'
terraform apply

Source: https://developer.hashicorp.com/terraform/language/values/variables#variables-on-the-command-line https://developer.hashicorp.com/terraform/language/values/variables#complex-typed-values

Repeated comment

To preserve Terraform Input Variable formatting, to create a *.tfvars and use operator -var-file=.

Given that a .tfplan is always generated using this Terraform Template execution Ansible Module, use of temporary .tfvars should at least be offered as an operator for the Ansible Module in addition to complex_vars: true ? For example, use_temporary_tfvars: true ?

anazobec commented 1 year ago

Hi, I have gone through your example code and tested it on my system. I've tried to reproduce your error using older versions of python (3.7) and different versions of this collection (1.0.0, 1.0.1), but the error isn't reproducible. I've also noticed that your playbook example above doesn't include a task named Terraform Module execution 2 as shown in your error output. There is also no example code for Terraform Module 2. If you want us to reproduce this error, please provide a minimal reproducible example.