search

ansible-collections / cloud.terraform

The collection automates the management and provisioning of infrastructure as code (IaC) using the Terraform CLI tool within Ansible playbooks and Execution Environment runtimes.
GNU General Public License v3.0
100 stars 36 forks source link

Can't read inventory from remote http state with terraform_state #166

Open Lyannic opened 2 months ago

Lyannic commented 2 months ago
SUMMARY

I can't read the inventory with the terraform_state plugin from a remote Terraform state that is stored at GitLab. The resources are created with the Hetzner Terraform provider. I don't get any error messages, but the resulting inventory is just empty.

ISSUE TYPE
COMPONENT NAME

cloud.terraform.terraform_state

ANSIBLE VERSION
ansible [core 2.17.3]
  config file = None
  configured module search path = ['/home/yannic/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/yannic/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/yannic/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/yannic/.local/bin/ansible
  python version = 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.1.3
  libyaml = True
COLLECTION VERSION
# /home/yannic/.ansible/collections/ansible_collections
Collection      Version   
--------------- ----------
cloud.terraform 4.0.0-dev0
CONFIGURATION
CONFIG_FILE() = None
OS / ENVIRONMENT

Ubuntu 22.04.4 LTS

STEPS TO REPRODUCE

I created a minimal example for my use-case.

main.tf

resource "hcloud_server" "server" {
  name         = "dyn-inv-test-server"
  image        = "docker-ce"
  server_type  = "cx22"
  location     = "fsn1"
}

resource "ansible_group" "docker_host_group" {
  name     = "docker_host"
}

resource "ansible_host" "server" {
  name   = hcloud_server.server.ipv4_address
  groups = [ansible_group.docker_host_group.name]
}

providers.tf

terraform {
  required_providers {
    hcloud = {
      source  = "hetznercloud/hcloud"
      version = "1.47.0"
    }
    ansible = {
      source = "ansible/ansible"
      version = "1.3.0"
    }
  }
}

provider "hcloud" {
  token = var.hcloud_token
}

backend.tf

terraform {
  backend "http" {
  }
}

terraform_state.yml

plugin: cloud.terraform.terraform_state
binary_path: /usr/bin/tofu
backend_type: http
search_child_modules: true
backend_config:
  address: https://gitlab.com/api/v4/projects/CENSORED/terraform/state/default
  lock_address: https://gitlab.com/api/v4/projects/CENSORED/terraform/state/default/lock
  unlock_address: https://gitlab.com/api/v4/projects/CENSORED/terraform/state/default/lock
  lock_method: POST
  unlock_method: DELETE
  retry_wait_min: 5
  username: CENSORED
  password: CENSORED
EXPECTED RESULTS

An inventory with the server that I created in main.tf. Here is the result that I get from cloud.terraform.terraform_provider

terraform_provider.yml

plugin: cloud.terraform.terraform_provider
binary_path: /usr/bin/tofu
ansible-inventory [core 2.17.3]
  config file = None
  configured module search path = ['/home/yannic/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/yannic/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/yannic/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/yannic/.local/bin/ansible-inventory
  python version = 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.1.3
  libyaml = True
No config file found; using defaults
setting up inventory plugins
Loading collection ansible.builtin from 
host_list declined parsing /home/yannic/projects/server-provisioning-dynamic-inventory-test/terraform_provider.yml as it did not pass its verify_file() method
script declined parsing /home/yannic/projects/server-provisioning-dynamic-inventory-test/terraform_provider.yml as it did not pass its verify_file() method
Loading collection cloud.terraform from /home/yannic/.ansible/collections/ansible_collections/cloud/terraform
Using inventory plugin 'ansible_collections.cloud.terraform.plugins.inventory.terraform_provider' to process inventory source '/home/yannic/projects/server-provisioning-dynamic-inventory-test/terraform_provider.yml'
Parsed /home/yannic/projects/server-provisioning-dynamic-inventory-test/terraform_provider.yml inventory source with auto plugin
{
    "_meta": {
        "hostvars": {}
    },
    "all": {
        "children": [
            "ungrouped",
            "docker_host"
        ]
    },
    "docker_host": {
        "hosts": [
            "76.32.12.154"
        ]
    }
}

Since I don't specify a state_file, terraform_provider is using the remote state too, which has the same config values as terraform_state.yml

terraform.tfstate

{
    "version": 3,
    "serial": 1,
    "lineage": "949ca244-3db7-4aaf-c5e5-03fb7126674b",
    "backend": {
        "type": "http",
        "config": {
            "address": "https://gitlab.com/api/v4/projects/CENSORED/terraform/state/default",
            "client_ca_certificate_pem": null,
            "client_certificate_pem": null,
            "client_private_key_pem": null,
            "headers": null,
            "lock_address": "https://gitlab.com/api/v4/projects/CENSORED/terraform/state/default/lock",
            "lock_method": "POST",
            "password": "CENSORED",
            "retry_max": null,
            "retry_wait_max": null,
            "retry_wait_min": 5,
            "skip_cert_verification": null,
            "unlock_address": "https://gitlab.com/api/v4/projects/CENSORED/terraform/state/default/lock",
            "unlock_method": "DELETE",
            "update_method": null,
            "username": "CENSORED"
        },
        "hash": 2984870146
    },
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {},
            "depends_on": []
        }
    ]
}
ACTUAL RESULTS

output from ansible-inventory -vvvvvv -i terraform_state.yml --list

ansible-inventory [core 2.17.3]
  config file = None
  configured module search path = ['/home/yannic/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/yannic/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/yannic/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/yannic/.local/bin/ansible-inventory
  python version = 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.1.3
  libyaml = True
No config file found; using defaults
setting up inventory plugins
Loading collection ansible.builtin from 
host_list declined parsing /home/yannic/projects/server-provisioning-dynamic-inventory-test/terraform_state.yml as it did not pass its verify_file() method
script declined parsing /home/yannic/projects/server-provisioning-dynamic-inventory-test/terraform_state.yml as it did not pass its verify_file() method
Loading collection cloud.terraform from /home/yannic/.ansible/collections/ansible_collections/cloud/terraform
Using inventory plugin 'ansible_collections.cloud.terraform.plugins.inventory.terraform_state' to process inventory source '/home/yannic/projects/server-provisioning-dynamic-inventory-test/terraform_state.yml'
Parsed /home/yannic/projects/server-provisioning-dynamic-inventory-test/terraform_state.yml inventory source with auto plugin
{
    "_meta": {
        "hostvars": {}
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    }
}
dr-acul commented 2 months ago

I think you are using a custom provider without specifying the provider_mapping (#146).

Adding the following code to my terraform_state.yml made it partially work:

hostnames:
  - ipv4_address
provider_mapping:
  - provider_name: registry.opentofu.org/hetznercloud/hcloud
    types:
      - hcloud_server

This does not use the ansible_host resource, and you probably have to play around with the mapping.