ec2_vpc_endpoint - InvalidVpcEndpointId.NotFound on creation

jatorcasso commented 2 years ago

Summary

While running integration tests for ec2_vpc_route_table (https://github.com/ansible-collections/amazon.aws/pull/690), I got an InvalidVpcEndpointId.NotFound error. After adding the parameter wait: yes, I no longer ran into the issue, but I feel that should be the default value on creation, rather than having to explicitly setting a wait parameter. Thoughts?

Issue Type

Bug Report

Component Name

ec2_vpc_endpoint

Ansible Version

$ ansible --version
ansible [core 2.12.1]
  config file = None
  configured module search path = ['/home/josephtorcasso/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/josephtorcasso/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/josephtorcasso/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/josephtorcasso/.local/bin/ansible
  python version = 3.10.0 (default, Oct  4 2021, 00:00:00) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)]
  jinja version = 3.0.1
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

AWS SDK versions

$ pip show boto boto3 botocore

Configuration

$ ansible-config dump --only-changed

OS / Environment

No response

Steps to Reproduce

  - name: create a VPC endpoint to test ec2_vpc_route_table ignores it
    ec2_vpc_endpoint:
      state: present
      vpc_id: '{{ vpc.vpc.id }}'
      service: com.amazonaws.{{ aws_region }}.s3
      route_table_ids:
      - '{{ recreate_private_table.route_table.route_table_id }}'
      # wait: yes
    register: vpc_endpoint

Expected Results

Expected a VPC endpoint to be created

Actual Results

Traceback (most recent call last):
  File "/tmp/ansible_ec2_vpc_endpoint_payload_24m4f4u0/ansible_ec2_vpc_endpoint_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_endpoint.py", line 251, in get_endpoints
  File "/tmp/ansible_ec2_vpc_endpoint_payload_24m4f4u0/ansible_ec2_vpc_endpoint_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/modules.py", line 328, in deciding_wrapper
    return retrying_wrapper(*args, **kwargs)
  File "/tmp/ansible_ec2_vpc_endpoint_payload_24m4f4u0/ansible_ec2_vpc_endpoint_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 118, in _retry_wrapper
    return _retry_func(
  File "/tmp/ansible_ec2_vpc_endpoint_payload_24m4f4u0/ansible_ec2_vpc_endpoint_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 68, in _retry_func
    return func()
  File "/usr/local/lib/python3.9/dist-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/dist-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidVpcEndpointId.NotFound) when calling the DescribeVpcEndpoints operation: The Vpc Endpoint Id 'vpce-0579edf0c8262f468' does not exist
fatal: [testhost]: FAILED! => {
    "boto3_version": "1.17.0",
    "botocore_version": "1.20.0",
    "changed": false,
    "error": {
        "code": "InvalidVpcEndpointId.NotFound",
        "message": "The Vpc Endpoint Id 'vpce-0579edf0c8262f468' does not exist"
    },
    "invocation": {
        "module_args": {
            "aws_access_key": "AKIA2PYXPFVF6H36TT5N",
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "client_token": null,
            "debug_botocore_endpoint_logs": true,
            "ec2_url": null,
            "policy": null,
            "policy_file": null,
            "profile": null,
            "purge_tags": false,
            "region": "us-east-1",
            "route_table_ids": [
                "rtb-03c42f3d9faf25b96"
            ],
            "security_token": null,
            "service": "com.amazonaws.us-east-1.s3",
            "state": "present",
            "tags": null,
            "validate_certs": true,
            "vpc_endpoint_id": null,
            "vpc_endpoint_security_groups": null,
            "vpc_endpoint_subnets": null,
            "vpc_endpoint_type": "Gateway",
            "vpc_id": "vpc-0c2009e4cc8209274",
            "wait": false,
            "wait_timeout": 320
        }
    },
    "msg": "Failed to get endpoints: An error occurred (InvalidVpcEndpointId.NotFound) when calling the DescribeVpcEndpoints operation: The Vpc Endpoint Id 'vpce-0579edf0c8262f468' does not exist",
    "resource_actions": [
        "ec2:DescribeVpcEndpoints",
        "ec2:CreateVpcEndpoint"
    ],
    "response_metadata": {
        "http_headers": {
            "cache-control": "no-cache, no-store",
            "connection": "close",
            "content-type": "text/xml;charset=UTF-8",
            "date": "Fri, 04 Mar 2022 15:51:40 GMT",
            "server": "AmazonEC2",
            "strict-transport-security": "max-age=31536000; includeSubDomains",
            "transfer-encoding": "chunked",
            "vary": "accept-encoding",
            "x-amzn-requestid": "7edb785f-3527-4020-bd25-908781ab02fb"
        },
        "http_status_code": 400,
        "request_id": "7edb785f-3527-4020-bd25-908781ab02fb",
        "retry_attempts": 0
    }
}

Code of Conduct

[X] I agree to follow the Ansible Code of Conduct

ansibullbot commented 2 years ago

Files identified in the description:

[plugins/modules/ec2_vpc_endpoint.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/ec2_vpc_endpoint.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 2 years ago

cc @Etherdaemon @jillr @s-hertel @tremble click here for bot help

jillr commented 2 years ago

Strictly, if wait=False we should not wait. We could do something like change https://github.com/ansible-collections/amazon.aws/blob/main/plugins/modules/ec2_vpc_endpoint.py#L379 to a try block, and in the case where the error we catch is InvalidVpcEndpointId.NotFound and wait=False we just return without any endpoint details. I don't like this solution though, personally.

We could also make get_endpoints optionally take extra error codes, include them in the params if they exist, and in the call at line 379 pass in catch_extra_error_codes=[InvalidVpcEndpointId.NotFound]. @tremble, you're using VPC Endpoints I believe, any thoughts here?

ETA: the second solution here is still technically a wait, just by a different name. So I guess it's more, do we want to change this behaviour or return an empty result?

jatorcasso commented 2 years ago

Is there an issue with defaulting wait=True?

jillr commented 2 years ago

There's nothing inherently wrong with doing the wait, but it would be a change in the default behaviour of the module so we would have to treat it as breaking. We can do that, conventionally we're planning for a new major release, but we should have consensus.

ansible-collections / amazon.aws