search

ansible-collections / community.aws

Ansible Collection for Community AWS
GNU General Public License v3.0
189 stars 398 forks source link

route53 cannot delete A record set #800

Closed kaovilai closed 2 years ago

kaovilai commented 2 years ago

Summary

Cannot delete A record without or with TTL value Providing TTL=null results in

An error occurred (InvalidInput) when calling the ChangeResourceRecordSets operation: Invalid request: Expected exactly one of [AliasTarget, all of [TTL, and ResourceRecords], or TrafficPolicyInstanceId], but found none in Change with [Action=DELETE, Name=<>., Type=A, SetIdentifier=null]

When not providing TTL, module uses default of 3600 Providing default ttl result in

An error occurred (InvalidChangeBatch) when calling the ChangeResourceRecordSets operation: [Tried to delete resource record set [name='<>.', type='A'] but the rdata provided is invalid]

Relevant blogs: https://blog.ryanhalliday.com/2019/09/route53-aliastarget-invalid-request.html

Issue Type

Bug Report

Component Name

route53

Ansible Version

$ ansible --version
ansible [core 2.11.3] 
  config file = None
  configured module search path = ['/Users/tiger/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/tiger/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.7 (default, Sep  3 2021, 12:37:55) [Clang 12.0.5 (clang-1205.0.22.9)]
  jinja version = 3.0.1
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
lections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0  
ansible.netcommon             2.2.0  
ansible.posix                 1.2.0  
ansible.utils                 2.3.0  
ansible.windows               1.7.0  
arista.eos                    2.2.0  
awx.awx                       19.2.2 
azure.azcollection            1.7.0  
check_point.mgmt              2.0.0  
chocolatey.chocolatey         1.1.0  
cisco.aci                     2.0.0  
cisco.asa                     2.0.2  
cisco.intersight              1.0.15 
cisco.ios                     2.3.0  
cisco.iosxr                   2.3.0  
cisco.meraki                  2.4.2  
cisco.mso                     1.2.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.4.0  
cisco.ucs                     1.6.0  
cloudscale_ch.cloud           2.2.0  
community.aws                 1.5.0  
community.azure               1.0.0  
community.crypto              1.7.1  
community.digitalocean        1.7.0  
community.docker              1.8.0  
community.fortios             1.0.0  
community.general             3.3.0  
community.google              1.0.0  
community.grafana             1.2.1  
community.hashi_vault         1.3.0  
community.hrobot              1.1.1  
community.kubernetes          1.2.1  
community.kubevirt            1.0.0  
community.libvirt             1.0.1  
community.mongodb             1.2.1  
community.mysql               2.1.0  
community.network             3.0.0  
community.okd                 1.1.2  
community.postgresql          1.3.0  
community.proxysql            1.0.0  
community.rabbitmq            1.0.3  
community.routeros            1.2.0  
community.skydive             1.0.0  
community.sops                1.1.0  
community.vmware              1.11.0 
community.windows             1.5.0  
community.zabbix              1.3.0  
containers.podman             1.6.1  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.7  
dellemc.enterprise_sonic      1.1.0  
dellemc.openmanage            3.5.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.10.1 
fortinet.fortimanager         2.1.2  
fortinet.fortios              2.1.1  
frr.frr                       1.0.3  
gluster.gluster               1.0.1  
google.cloud                  1.0.2  
hetzner.hcloud                1.4.3  
hpe.nimble                    1.1.3  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.2.4  
inspur.sm                     1.2.0  
junipernetworks.junos         2.3.0  
kubernetes.core               1.2.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.2.0 
netapp.azure                  21.7.0 
netapp.cloudmanager           21.7.0 
netapp.elementsw              21.6.1 
netapp.ontap                  21.7.0 
netapp.um_info                21.6.0 
netapp_eseries.santricity     1.2.13 
netbox.netbox                 3.1.1  
ngine_io.cloudstack           2.1.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.0  
openstack.cloud               1.5.0  
openvswitch.openvswitch       2.0.0  
ovirt.ovirt                   1.5.3  
purestorage.flasharray        1.8.0  
purestorage.flashblade        1.6.0  
sensu.sensu_go                1.11.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.18.0 
theforeman.foreman            2.1.1  
vyos.vyos                     2.3.1  
wti.remote                    1.0.1

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /usr/local/lib/python3.9/site-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.14.43
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: UNKNOWN
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.17.43
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: UNKNOWN
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: docutils, jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

OS / Environment

macos

Steps to Reproduce

  - name: Delete openshift api route53 A record
    community.aws.route53:
      state: absent
      aws_access_key: "{{ aws_access_key_id }}"
      aws_secret_key: "{{ aws_secret_access_key }}"
      zone: "{{ guid }}.{{ aws_dns_zone_root }}"
      record: "{{ records.set.record }}"
      value: "{{ records.set.value }}"
      ttl: null
      type: "{{ records.set.type }}"

or leave empty ttl.. results in default ttl 3600 used

  - name: Delete openshift api route53 A record
    community.aws.route53:
      state: absent
      aws_access_key: "{{ aws_access_key_id }}"
      aws_secret_key: "{{ aws_secret_access_key }}"
      zone: "{{ guid }}.{{ aws_dns_zone_root }}"
      record: "{{ records.set.record }}"
      value: "{{ records.set.value }}"
      type: "{{ records.set.type }}"

Expected Results

A record is deleted without TTL errors

Actual Results

ttl = null

The full traceback is:
Traceback (most recent call last):
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_community.aws.route53_payload_c4b6fh52/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 652, in main
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_community.aws.route53_payload_c4b6fh52/ansible_community.aws.route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 288, in deciding_wrapper
    return retrying_wrapper(*args, **kwargs)
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_community.aws.route53_payload_c4b6fh52/ansible_community.aws.route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 154, in retry_func
    raise e
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_community.aws.route53_payload_c4b6fh52/ansible_community.aws.route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 144, in retry_func
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidInput: An error occurred (InvalidInput) when calling the ChangeResourceRecordSets operation: Invalid request: Expected exactly one of [AliasTarget, all of [TTL, and ResourceRecords], or TrafficPolicyInstanceId], but found none in Change with [Action=DELETE, Name=<>., Type=A, SetIdentifier=null]
fatal: [localhost]: FAILED! => {
    "boto3_version": "1.14.43",
    "botocore_version": "1.17.43",
    "changed": false,
    "error": {
        "code": "InvalidInput",
        "message": "Invalid request: Expected exactly one of [AliasTarget, all of [TTL, and ResourceRecords], or TrafficPolicyInstanceId], but found none in Change with [Action=DELETE, Name=<>., Type=A, SetIdentifier=null]",
        "type": "Sender"
    },
    "invocation": {
        "module_args": {
            "alias": null,
            "alias_evaluate_target_health": false,
            "alias_hosted_zone_id": null,
            "aws_access_key": "<>",
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": "<>",
            "debug_botocore_endpoint_logs": false,
            "ec2_url": null,
            "failover": null,
            "health_check": null,
            "hosted_zone_id": null,
            "identifier": null,
            "overwrite": null,
            "private_zone": false,
            "profile": null,
            "record": "<>.",
            "region": null,
            "retry_interval": 500,
            "security_token": null,
            "state": "absent",
            "ttl": null,
            "type": "A",
            "validate_certs": true,
            "value": [
                "<>"
            ],
            "vpc_id": null,
            "wait": false,
            "wait_timeout": 300,
            "weight": null,
            "zone": "<>"
        }
    },
    "msg": "Failed to update records: An error occurred (InvalidInput) when calling the ChangeResourceRecordSets operation: Invalid request: Expected exactly one of [AliasTarget, all of [TTL, and ResourceRecords], or TrafficPolicyInstanceId], but found none in Change with [Action=DELETE, Name=<>., Type=A, SetIdentifier=null]",
    "response_metadata": {
        "http_headers": {
            "connection": "close",
            "content-length": "507",
            "content-type": "text/xml",
            "date": "Fri, 12 Nov 2021 14:41:35 GMT",
            "x-amzn-requestid": <>
        },
        "http_status_code": 400,
        "request_id": <>,
        "retry_attempts": 0
    }
}

ttl left to default

The full traceback is:
Traceback (most recent call last):
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_community.aws.route53_payload_tvoqyvr_/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 652, in main
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_community.aws.route53_payload_tvoqyvr_/ansible_community.aws.route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 288, in deciding_wrapper
    return retrying_wrapper(*args, **kwargs)
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_community.aws.route53_payload_tvoqyvr_/ansible_community.aws.route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 154, in retry_func
    raise e
  File "/var/folders/62/wkysd_4n0w57ljl9ycfsd9cc0000gn/T/ansible_community.aws.route53_payload_tvoqyvr_/ansible_community.aws.route53_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 144, in retry_func
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidChangeBatch: An error occurred (InvalidChangeBatch) when calling the ChangeResourceRecordSets operation: [Tried to delete resource record set [name='<>.', type='A'] but the rdata provided is invalid]
fatal: [localhost]: FAILED! => {
    "boto3_version": "1.14.43",
    "botocore_version": "1.17.43",
    "changed": false,
    "error": {
        "code": "InvalidChangeBatch",
        "message": "[Tried to delete resource record set [name='<>.', type='A'] but the rdata provided is invalid]",
        "type": "Sender"
    },
    "invocation": {
        "module_args": {
            "alias": null,
            "alias_evaluate_target_health": false,
            "alias_hosted_zone_id": null,
            "aws_access_key": "<>",
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "debug_botocore_endpoint_logs": false,
            "ec2_url": null,
            "failover": null,
            "health_check": null,
            "hosted_zone_id": null,
            "identifier": null,
            "overwrite": null,
            "private_zone": false,
            "profile": null,
            "record": "<>.",
            "region": null,
            "retry_interval": 500,
            "security_token": null,
            "state": "absent",
            "ttl": 3600,
            "type": "A",
            "validate_certs": true,
            "value": [
                "<>."
            ],
            "vpc_id": null,
            "wait": false,
            "wait_timeout": 300,
            "weight": null,
            "zone": "<>"
        }
    },
    "msg": "Failed to update records: An error occurred (InvalidChangeBatch) when calling the ChangeResourceRecordSets operation: [Tried to delete resource record set [name='<>.', type='A'] but the rdata provided is invalid]",
    "response_metadata": {
        "http_headers": {
            "connection": "close",
            "content-length": "405",
            "content-type": "text/xml",
            "date": "Fri, 12 Nov 2021 14:55:42 GMT",
            "x-amzn-requestid": "<>"
        },
        "http_status_code": 400,
        "request_id": "<>",
        "retry_attempts": 0
    }
}

Code of Conduct

kaovilai commented 2 years ago

Solution would be to make TTL nullable.. and if null.. don't provide TTL in the request.

markuman commented 2 years ago

I can confirm.
Basically when deleting records, the name and type are sufficient. Everything else does not matter. That's how community.dns is handling it imo.

However, just leaving it out makes botocore fail

 "msg": "Failed to update records: An error occurred (InvalidInput) when calling the ChangeResourceRecordSets operation: Invalid request: Expected exactly one of [AliasTarget, all of [TTL, and ResourceRecords], or TrafficPolicyInstanceId], but found none in Change with [Action=DELETE, Name=ansible.xn--mitlinuxwrdasnichtpassiert-ohc.de., Type=A, SetIdentifier=null]",

So we must keep the TTL of the existing record.
This is kind of a hotfix. But I need to make some more tests later.

diff --git a/plugins/modules/route53.py b/plugins/modules/route53.py
index 9640202..e971e18 100644
--- a/plugins/modules/route53.py
+++ b/plugins/modules/route53.py
@@ -596,17 +596,29 @@ def main():

     aws_record = get_record(route53, zone_id, record_in, type_in, identifier_in)

-    resource_record_set = scrub_none_parameters({
-        'Name': record_in,
-        'Type': type_in,
-        'Weight': weight_in,
-        'Region': region_in,
-        'Failover': failover_in,
-        'TTL': ttl_in,
-        'ResourceRecords': [dict(Value=value) for value in value_in],
-        'HealthCheckId': health_check_in,
-        'SetIdentifier': identifier_in,
-    })
+    if command_in is not 'delete':
+        resource_record_set = scrub_none_parameters({
+            'Name': record_in,
+            'Type': type_in,
+            'Weight': weight_in,
+            'Region': region_in,
+            'Failover': failover_in,
+            'TTL': ttl_in,
+            'ResourceRecords': [dict(Value=value) for value in value_in],
+            'HealthCheckId': health_check_in,
+            'SetIdentifier': identifier_in,
+        })
+    else:
+        resource_record_set = scrub_none_parameters({
+            'Name': record_in,
+            'Type': type_in,
+            'Weight': weight_in,
+            'Region': region_in,
+            'Failover': failover_in,
+            'ResourceRecords': [dict(Value=value) for value in value_in],
+            'HealthCheckId': health_check_in,
+            'TTL': aws_record.get('TTL')
+        })

     if alias_in:
         resource_record_set['AliasTarget'] = dict(