ec2_instace assuming Default Subnet instead of Launch Template specified one

paulogodinhoaq commented 2 years ago

Summary

When launching an instance from a template, the module will try to use the default subnet instead of the one set inside the actual template, conflicting with the Security Group set inside the template and failing to instantiate the machine.

Task:

- name: Spawn Machine
    amazon.aws.ec2_instance:
      state: running
      name: "[Spawned From Ansible] Test Micro Machine"
      instance_initiated_shutdown_behavior: terminate
      launch_template:
        id: "lt-0c7678a944ceeac69"
        version: 3

Run Command: ansible-playbook setup_machine.yml --connection=local -vvv

The default subnet is used in the call as it is visible bellow, it should not send a subnet unless one is specified as the template already has one defined.

This Launch Template works without issue through the AWS CLI directly

Issue Type

Bug Report

Component Name

ec2_instance

Ansible Version

  config file = None
  configured module search path = ['/home/phgdodinho/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/phgdodinho/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/phgdodinho/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/phgdodinho/.local/bin/ansible
  python version = 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
  jinja version = 3.0.3
  libyaml = True

Collection Versions

----------------------------- -------
amazon.aws                    2.2.0
ansible.netcommon             2.6.1
ansible.posix                 1.3.0
ansible.utils                 2.6.1
ansible.windows               1.10.0
arista.eos                    3.1.0
awx.awx                       19.4.0
azure.azcollection            1.12.0
check_point.mgmt              2.3.0
chocolatey.chocolatey         1.2.0
cisco.aci                     2.2.0
cisco.asa                     2.1.0
cisco.intersight              1.0.18
cisco.ios                     2.8.1
cisco.iosxr                   2.9.0
cisco.ise                     1.2.1
cisco.meraki                  2.6.2
cisco.mso                     1.4.0
cisco.nso                     1.0.3
cisco.nxos                    2.9.1
cisco.ucs                     1.8.0
cloud.common                  2.1.1
cloudscale_ch.cloud           2.2.1
community.aws                 2.4.0
community.azure               1.1.0
community.ciscosmb            1.0.5
community.crypto              2.3.1
community.digitalocean        1.19.0
community.dns                 2.1.1
community.docker              2.5.1
community.fortios             1.0.0
community.general             4.8.1
community.google              1.0.0
community.grafana             1.4.0
community.hashi_vault         2.5.0
community.hrobot              1.3.0
community.kubernetes          2.0.1
community.kubevirt            1.0.0
community.libvirt             1.1.0
community.mongodb             1.4.0
community.mysql               2.3.7
community.network             3.3.0
community.okd                 2.2.0
community.postgresql          1.7.4
community.proxysql            1.3.2
community.rabbitmq            1.2.1
community.routeros            2.0.0
community.sap                 1.0.0
community.skydive             1.0.0
community.sops                1.2.1
community.vmware              1.18.0
community.windows             1.10.0
community.zabbix              1.6.0
containers.podman             1.9.3
cyberark.conjur               1.1.0
cyberark.pas                  1.0.13
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            4.4.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.16.0
fortinet.fortimanager         2.1.5
fortinet.fortios              2.1.4
frr.frr                       1.0.4
gluster.gluster               1.0.2
google.cloud                  1.0.2
hetzner.hcloud                1.6.0
hpe.nimble                    1.1.4
ibm.qradar                    1.0.3
infinidat.infinibox           1.3.3
infoblox.nios_modules         1.2.1
inspur.sm                     1.3.0
junipernetworks.junos         2.10.0
kubernetes.core               2.3.1
mellanox.onyx                 1.0.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.17.0
netapp.elementsw              21.7.0
netapp.ontap                  21.19.1
netapp.storagegrid            21.10.0
netapp.um_info                21.8.0
netapp_eseries.santricity     1.3.0
netbox.netbox                 3.7.1
ngine_io.cloudstack           2.2.3
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.1
openstack.cloud               1.8.0
openvswitch.openvswitch       2.1.0
ovirt.ovirt                   1.6.6
purestorage.flasharray        1.13.0
purestorage.flashblade        1.9.0
sensu.sensu_go                1.13.1
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.29.0
theforeman.foreman            2.2.0
vmware.vmware_rest            2.1.5
vyos.vyos                     2.8.0
wti.remote                    1.0.3

AWS SDK versions

Name: boto3
Version: 1.23.3
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/phgdodinho/.local/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: awsume
---
Name: botocore
Version: 1.26.3
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/phgdodinho/.local/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

OS / Environment

Ubuntu 22.04 LTS on WSL - Windows 10 x86_64

Steps to Reproduce

Task:

- name: Spawn Machine
    amazon.aws.ec2_instance:
      state: running
      name: "[Spawned From Ansible] Test Micro Machine"
      instance_initiated_shutdown_behavior: terminate
      launch_template:
        id: "lt-0c7678a944ceeac69"
        version: 3

Run Command: ansible-playbook setup_machine.yml --connection=local -vvv

Expected Results

Machine is created

Actual Results

It is for some reason it is sending the default subnet "subnet-id": "subnet-008c094dfe760bda4" inside the request which conflicts with the Security Group set in the template, failing to create the machine.


Traceback (most recent call last):
  File "/tmp/ansible_amazon.aws.ec2_instance_payload_l15t8c2t/ansible_amazon.aws.ec2_instance_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_instance.py", line 1816, in ensure_present
  File "/tmp/ansible_amazon.aws.ec2_instance_payload_l15t8c2t/ansible_amazon.aws.ec2_instance_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_instance.py", line 1865, in run_instances
  File "/tmp/ansible_amazon.aws.ec2_instance_payload_l15t8c2t/ansible_amazon.aws.ec2_instance_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 336, in deciding_wrapper
    return unwrapped(*args, **kwargs)
  File "/home/phgdodinho/.local/lib/python3.10/site-packages/botocore/client.py", line 508, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/phgdodinho/.local/lib/python3.10/site-packages/botocore/client.py", line 911, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidParameter) when calling the RunInstances operation: Security group sg-0d595415951402832 and subnet subnet-008c094dfe760bda4 belong to different networks.
fatal: [localhost]: FAILED! => {
    "boto3_version": "1.23.3",
    "botocore_version": "1.26.3",
    "changed": false,
    "error": {
        "code": "InvalidParameter",
        "message": "Security group sg-0d595415951402832 and subnet subnet-008c094dfe760bda4 belong to different networks."
    },
    "invocation": {
        "module_args": {
            "availability_zone": null,
            "aws_access_key": null,
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": null,
            "count": null,
            "cpu_credit_specification": null,
            "cpu_options": null,
            "debug_botocore_endpoint_logs": false,
            "detailed_monitoring": null,
            "ebs_optimized": null,
            "ec2_url": null,
            "exact_count": null,
            "filters": {
                "instance-state-name": [
                    "pending",
                    "running",
                    "stopping",
                    "stopped"
                ],
                "subnet-id": "subnet-008c094dfe760bda4",
                "tag:Name": [
                    "[Spawned From Ansible] Test Micro Machine"
                ]
            },
            "image": null,
            "image_id": null,
            "instance_ids": [],
            "instance_initiated_shutdown_behavior": "terminate",
            "instance_role": null,
            "instance_type": "t2.micro",
            "key_name": null,
            "launch_template": {
                "id": "lt-0c7678a944ceeac69",
                "version": 3
            },
            "metadata_options": null,
            "name": "[Spawned From Ansible] Test Micro Machine",
            "network": null,
            "placement_group": null,
            "profile": null,
            "purge_tags": false,
            "region": null,
            "security_group": null,
            "security_groups": [],
            "security_token": null,
            "state": "running",
            "tags": null,
            "tenancy": null,
            "termination_protection": null,
            "tower_callback": null,
            "user_data": null,
            "validate_certs": true,
            "volumes": null,
            "vpc_subnet_id": null,
            "wait": true,
            "wait_timeout": 600
        }
    },
    "msg": "Failed to create new EC2 instance: An error occurred (InvalidParameter) when calling the RunInstances operation: Security group sg-0d595415951402832 and subnet subnet-008c094dfe760bda4 belong to different networks.",
    "response_metadata": {
        "http_headers": {
            "cache-control": "no-cache, no-store",
            "connection": "close",
            "content-type": "text/xml;charset=UTF-8",
            "date": "Thu, 19 May 2022 21:40:41 GMT",
            "server": "AmazonEC2",
            "strict-transport-security": "max-age=31536000; includeSubDomains",
            "transfer-encoding": "chunked",
            "vary": "accept-encoding",
            "x-amzn-requestid": "20f5cbf9-62c4-48d7-816d-49571eb82956"
        },
        "http_status_code": 400,
        "request_id": "20f5cbf9-62c4-48d7-816d-49571eb82956",
        "retry_attempts": 0
    }
}```

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct

ansibullbot commented 2 years ago

Files identified in the description:

[plugins/modules/ec2_instance.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/ec2_instance.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 2 years ago

cc @jillr @ryansb @s-hertel @tremble click here for bot help

paulogodinhoaq commented 2 years ago

I have confirmed that manually setting the vpc_subnet_id is an workaround, but I believe the module should not imply default subnet when dealing with templates.

This works:

  - name: Spawn Machine
    amazon.aws.ec2_instance:
      state: running
      name: "[Spawned From Ansible] Test Micro Machine"
      vpc_subnet_id: 'subnet-0c5a581319a69910a'
      launch_template:
        id: "lt-0c7678a944ceeac69"
        version: 3

Cheers.

paulogodinhoaq commented 2 years ago

It looks like the "instance_type": "t2.micro" is also being sent, even if the template has the instance type properly set. The workaround is to also send the matching instance_type in the module usage.

Only way to currently use Launch Templates currently, specifying both vpc_subnet_id and instace_type

  - name: Spawn Machine
    amazon.aws.ec2_instance:
      state: running
      name: "[Spawned From Ansible] Windows for Custom Unreal AMI Creation"
      vpc_subnet_id: 'subnet-0c5a581319a69910a'
      instance_type: 'c5.2xlarge'
      launch_template:
        id: "lt-03714509be6768d02"
        version: 2
    register: ec2spawnedmachines

goneri commented 2 years ago

Hi @paulogodinhoaq,

You're using a rather oudated copy of amazon.aws. May I ask you to update, just to validate the behaviour you describe is still the same.

paulogodinhoaq commented 2 years ago

Hi @paulogodinhoaq,

You're using a rather oudated copy of amazon.aws. May I ask you to update, just to validate the behaviour you describe is still the same.

Interesting, I had just recently installed ansible using pip, I thought the "packages" would come with the latest version, what would be the correct way to update it?

Cheers

abikouo commented 2 years ago

Hi @paulogodinhoaq, You're using a rather oudated copy of amazon.aws. May I ask you to update, just to validate the behaviour you describe is still the same.

Interesting, I had just recently installed ansible using pip, I thought the "packages" would come with the latest version, what would be the correct way to update it?

Cheers

Hi @paulogodinhoaq

Try using ansible-galaxy collection install git+https://github.com/ansible-collections/amazon.aws.git

ansible-collections / amazon.aws