`cloud.terraform.terraform_state` Plugin Fails to Include EC2 Instances Created via Terraform Modules in Ansible Inventory

[kedar9696]

SUMMARY

The cloud.terraform.terraform_state plugin fails to retrieve AWS EC2 instances that are created via Terraform modules when generating an Ansible inventory. Instances defined directly using the aws_instance resource type are successfully fetched, but those within modules (including both custom and third-party modules) are not recognized by the plugin.

ISSUE TYPE

• Bug Report

COMPONENT NAME

cloud.terraform.terraform_state plugin

ANSIBLE VERSION

ansible [core 2.17.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/devops/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/devops/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/devops/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

COLLECTION VERSION

# /home/devops/.ansible/collections/ansible_collections
Collection      Version
--------------- -------
cloud.terraform 3.0.0  

CONFIGURATION

CALLBACKS_ENABLED(/etc/ansible/ansible.cfg) = ['profile_tasks']
CONFIG_FILE() = /etc/ansible/ansible.cfg
DEFAULT_ASK_PASS(/etc/ansible/ansible.cfg) = False
DEFAULT_BECOME(/etc/ansible/ansible.cfg) = True
DEFAULT_BECOME_ASK_PASS(/etc/ansible/ansible.cfg) = False
DEFAULT_BECOME_METHOD(/etc/ansible/ansible.cfg) = sudo
DEFAULT_BECOME_USER(/etc/ansible/ansible.cfg) = root
DEFAULT_PRIVATE_KEY_FILE(/etc/ansible/ansible.cfg) = /var/lib/rundeck/devops.pem
DEFAULT_REMOTE_USER(/etc/ansible/ansible.cfg) = dev.ops
DEFAULT_ROLES_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/roles']
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
INVENTORY_ENABLED(/etc/ansible/ansible.cfg) = ['host_list', 'script', 'auto', 'yaml', 'ini', 'toml', 'azure_rm', 'aws_ec2']

OS / ENVIRONMENT

No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.4 LTS Release: 22.04 Codename: jammy

STEPS TO REPRODUCE

Define AWS EC2 instances using three different methods in Terraform: Directly using aws_instance resource. Using a custom module (module "ec2") that internally defines aws_instance. Using a third-party module (module "ec2_instance") like terraform-aws-modules/ec2-instance/aws. Apply the Terraform configuration to create resources. Use the cloud.terraform.terraform_state plugin to generate an Ansible inventory.

# main.tf

terraform {
  required_version = "1.9.3"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.64.0"
    }
    ansible = {
      source  = "registry.terraform.io/ansible/ansible"
      version = "1.3.0"
    }
  }
  backend "s3" {
    bucket     = "cognis"
    key        = "terraform/devops/terraform.tfstate"
    region     = "ap-south-1"

provider "aws" {
  region     = "ap-south-1"
}

resource "aws_instance" "web" {
  ami           = "ami-02b49a24cfb95941c"
  key_name      = "cognis-aws-keys"
  instance_type = "t2.micro"

  tags = {
    Name = "HelloWorld"
  }
}

module "ec2" {
  source         = "../modules/ec2"
  instance_type  = "t2.micro"
  instance_count = 1
  key_name              = "cognis-aws-keys"
  public_ip             = true
  ebs_volume_size       = 8
  ebs_volume_type       = "gp3"
  ebs_iops              = 3000
  ebs_throughput        = 125
  delete_on_termination = true
  name                  = "Module-Server"
  ec2_tags              = { Name = "DB-server-1" }
}

module "ec2_instance" {
  source = "terraform-aws-modules/ec2-instance/aws"

  name          = "single-instance"
  ami           = "ami-02b49a24cfb95941c"
  instance_type = "t2.micro"
  key_name      = "cognis-aws-keys"
  monitoring    = false

  tags = {
    Terraform   = "true"
    Environment = "dev"
    Name        = "registry_module"
  }
}

# ansible_terraform_state.yaml
plugin: cloud.terraform.terraform_state
backend_type: s3
backend_config:
  region: ap-south-1
  key: terraform/devops/terraform.tfstate
  bucket: cognis

EXPECTED RESULTS

The Ansible inventory should include all EC2 instances created by Terraform, regardless of whether they were defined directly or within modules.

ACTUAL RESULTS

Only the EC2 instances defined directly using the aws_instance resource type are included in the Ansible inventory. Instances created within custom or third-party modules are not recognized or included.

$ ansible-inventory -i ansible_terraform_state.yaml --graph -vvvv

ansible-inventory [core 2.17.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.10/site-packages/ansible
  executable location = /usr/bin/ansible-inventory
  python version = 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

Using inventory plugin 'cloud.terraform.terraform_state' to process inventory source 'ansible_terraform_state.yaml'
Parsed ansible_terraform_state.yaml inventory source with auto plugin
@all:
  |--@ungrouped:
  |  |--HelloWorld

# Instances from modules are missing

[abikouo]

@kedar9696 you need to set the parameter search_child_modules: true in order for the plugin to include modules resources

[kedar9696]

Thank you, @abikouo, for your help! Everything is working as expected now. I realized I missed that parameter; it was my mistake.