search

aws / aws-sdk

Landing page for the AWS SDKs on GitHub
https://aws.amazon.com/tools/
Other
72 stars 14 forks source link

Route53 updates only one CNAME per batch #472

Closed tim-finnigan closed 6 months ago

tim-finnigan commented 1 year ago

Original issue: https://github.com/boto/boto3/issues/3579

Related issue: https://github.com/octodns/octodns-route53/issues/45#issuecomment-1345664541

re:Post question: https://repost.aws/questions/QUj9pIHHnyRIGKNJqjqLgaZQ/route-53-batch-of-changes-succeeding-but-only-making-one-of-the-changes

tim-finnigan commented 1 year ago

P90892106

anemyte commented 1 year ago

Hi, Here's the debug log that you requested. There are three requests total:

  1. Creates test records
  2. Updates them
  3. Retrieves current state with list_resource_record_sets(), human-readable response below the log.

The only thing I removed from this is my aws_access_key_id.

2023-02-09 19:21:52,331 botocore.hooks [DEBUG] Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2023-02-09 19:21:52,331 botocore.hooks [DEBUG] Changing event name from before-call.apigateway to before-call.api-gateway
2023-02-09 19:21:52,331 botocore.hooks [DEBUG] Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2023-02-09 19:21:52,332 botocore.hooks [DEBUG] Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2023-02-09 19:21:52,332 botocore.hooks [DEBUG] Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2023-02-09 19:21:52,332 botocore.hooks [DEBUG] Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2023-02-09 19:21:52,332 botocore.hooks [DEBUG] Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2023-02-09 19:21:52,333 botocore.hooks [DEBUG] Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2023-02-09 19:21:52,333 botocore.hooks [DEBUG] Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2023-02-09 19:21:52,333 botocore.hooks [DEBUG] Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2023-02-09 19:21:52,333 botocore.hooks [DEBUG] Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2023-02-09 19:21:52,334 botocore.utils [DEBUG] IMDS ENDPOINT: http://169.254.169.254/
2023-02-09 19:21:52,335 botocore.credentials [DEBUG] Looking for credentials via: env
2023-02-09 19:21:52,335 botocore.credentials [DEBUG] Looking for credentials via: assume-role
2023-02-09 19:21:52,335 botocore.credentials [DEBUG] Looking for credentials via: assume-role-with-web-identity
2023-02-09 19:21:52,335 botocore.credentials [DEBUG] Looking for credentials via: sso
2023-02-09 19:21:52,335 botocore.credentials [DEBUG] Looking for credentials via: shared-credentials-file
2023-02-09 19:21:52,335 botocore.credentials [INFO] Found credentials in shared credentials file: ~/.aws/credentials
2023-02-09 19:21:52,337 botocore.loaders [DEBUG] Loading JSON file: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/data/endpoints.json
2023-02-09 19:21:52,342 botocore.loaders [DEBUG] Loading JSON file: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/data/sdk-default-configuration.json
2023-02-09 19:21:52,342 botocore.hooks [DEBUG] Event choose-service-name: calling handler <function handle_service_name_alias at 0x7f91216b6f80>
2023-02-09 19:21:52,346 botocore.loaders [DEBUG] Loading JSON file: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/data/route53/2013-04-01/service-2.json
2023-02-09 19:21:52,351 botocore.loaders [DEBUG] Loading JSON file: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/data/route53/2013-04-01/endpoint-rule-set-1.json.gz
2023-02-09 19:21:52,352 botocore.loaders [DEBUG] Loading JSON file: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/data/partitions.json
2023-02-09 19:21:52,352 botocore.hooks [DEBUG] Event creating-client-class.route-53: calling handler <function add_generate_presigned_url at 0x7f91215eb1c0>
2023-02-09 19:21:52,352 botocore.regions [DEBUG] Using partition endpoint for route53, eu-central-1: aws-global
2023-02-09 19:21:52,353 botocore.endpoint [DEBUG] Setting route53 timeout as (60, 60)
2023-02-09 19:21:52,353 botocore.loaders [DEBUG] Loading JSON file: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/data/_retry.json
2023-02-09 19:21:52,353 botocore.client [DEBUG] Registering retry handlers for service: route53
2023-02-09 19:21:52,353 botocore.regions [DEBUG] Calling endpoint provider with parameters: {'Region': 'eu-central-1', 'UseDualStack': False, 'UseFIPS': False}
2023-02-09 19:21:52,354 botocore.regions [DEBUG] Endpoint provider result: https://route53.amazonaws.com
2023-02-09 19:21:52,354 botocore.regions [DEBUG] Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None"
2023-02-09 19:21:52,354 botocore.regions [DEBUG] Selected auth type "v4" as "v4" with signing context params: {'region': 'us-east-1', 'signing_name': 'route53'}
2023-02-09 19:21:52,354 botocore.hooks [DEBUG] Event before-parameter-build.route-53.ChangeResourceRecordSets: calling handler <function fix_route53_ids at 0x7f91214dd360>
2023-02-09 19:21:52,354 botocore.handlers [DEBUG] HostedZoneId Z0720548IECD1KKM2UJH -> Z0720548IECD1KKM2UJH
2023-02-09 19:21:52,354 botocore.hooks [DEBUG] Event before-parameter-build.route-53.ChangeResourceRecordSets: calling handler <function generate_idempotent_uuid at 0x7f91214dc4c0>
2023-02-09 19:21:52,354 botocore.hooks [DEBUG] Event before-call.route-53.ChangeResourceRecordSets: calling handler <function add_recursion_detection_header at 0x7f91214dc160>
2023-02-09 19:21:52,354 botocore.hooks [DEBUG] Event before-call.route-53.ChangeResourceRecordSets: calling handler <function inject_api_version_header_if_needed at 0x7f91214ddcf0>
2023-02-09 19:21:52,354 botocore.endpoint [DEBUG] Making request for OperationModel(name=ChangeResourceRecordSets) with params: {'url_path': '/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/', 'query_string': {}, 'method': 'POST', 'headers': {'User-Agent': 'Boto3/1.26.64 Python/3.10.9 Linux/6.1.8-arch1-1 Botocore/1.29.64'}, 'body': b'<ChangeResourceRecordSetsRequest xmlns="https://route53.amazonaws.com/doc/2013-04-01/"><ChangeBatch><Comment>create test records</Comment><Changes><Change><Action>UPSERT</Action><ResourceRecordSet><Name>foo.pytestexample.com</Name><Type>CNAME</Type><TTL>123</TTL><ResourceRecords><ResourceRecord><Value>bar.pytestexample.com</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></Change><Change><Action>UPSERT</Action><ResourceRecordSet><Name>spam.pytestexample.com</Name><Type>CNAME</Type><TTL>123</TTL><ResourceRecords><ResourceRecord><Value>eggs.pytestexample.com</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></Change></Changes></ChangeBatch></ChangeResourceRecordSetsRequest>', 'url': 'https://route53.amazonaws.com/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/', 'context': {'client_region': 'aws-global', 'client_config': <botocore.config.Config object at 0x7f912141c400>, 'has_streaming_input': False, 'auth_type': 'v4', 'signing': {'region': 'us-east-1', 'signing_name': 'route53'}}}
2023-02-09 19:21:52,354 botocore.hooks [DEBUG] Event request-created.route-53.ChangeResourceRecordSets: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f912141c2b0>>
2023-02-09 19:21:52,354 botocore.hooks [DEBUG] Event choose-signer.route-53.ChangeResourceRecordSets: calling handler <function set_operation_specific_signer at 0x7f91214dc3a0>
2023-02-09 19:21:52,354 botocore.auth [DEBUG] Calculating signature using v4 auth.
2023-02-09 19:21:52,354 botocore.auth [DEBUG] CanonicalRequest:
POST
/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/

host:route53.amazonaws.com
x-amz-date:20230209T162152Z

host;x-amz-date
31f670557271efe991b311af7271a637680acce27607b51669d808679ae7ccfc
2023-02-09 19:21:52,354 botocore.auth [DEBUG] StringToSign:
AWS4-HMAC-SHA256
20230209T162152Z
20230209/us-east-1/route53/aws4_request
7b043a7acec5e324de3ba01502ea27188e4f653d16c7d79e3eb2c9803d9419e9
2023-02-09 19:21:52,354 botocore.auth [DEBUG] Signature:
613408a4b214e913bcf09dcdf6d55ea1c9f6cd6415e101e1e6d482ba60f8cff0
2023-02-09 19:21:52,354 botocore.hooks [DEBUG] Event request-created.route-53.ChangeResourceRecordSets: calling handler <function add_retry_headers at 0x7f91214de3b0>
2023-02-09 19:21:52,354 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://route53.amazonaws.com/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/, headers={'User-Agent': b'Boto3/1.26.64 Python/3.10.9 Linux/6.1.8-arch1-1 Botocore/1.29.64', 'X-Amz-Date': b'20230209T162152Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIADELETED/20230209/us-east-1/route53/aws4_request, SignedHeaders=host;x-amz-date, Signature=613408a4b214e913bcf09dcdf6d55ea1c9f6cd6415e101e1e6d482ba60f8cff0', 'amz-sdk-invocation-id': b'd9bdb380-d0fa-40a2-b671-d4755371a344', 'amz-sdk-request': b'attempt=1', 'Content-Length': '705'}>
2023-02-09 19:21:52,355 botocore.httpsession [DEBUG] Certificate path: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/cacert.pem
2023-02-09 19:21:52,355 urllib3.connectionpool [DEBUG] Starting new HTTPS connection (1): route53.amazonaws.com:443
2023-02-09 19:21:53,063 urllib3.connectionpool [DEBUG] https://route53.amazonaws.com:443 "POST /2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/ HTTP/1.1" 200 321
2023-02-09 19:21:53,063 botocore.parsers [DEBUG] Response headers: {'x-amzn-RequestId': '91c49009-aa29-4135-bbe8-490bac5143bc', 'Content-Type': 'text/xml', 'Content-Length': '321', 'Date': 'Thu, 09 Feb 2023 16:21:52 GMT'}
2023-02-09 19:21:53,063 botocore.parsers [DEBUG] Response body:
b'<?xml version="1.0"?>\n<ChangeResourceRecordSetsResponse xmlns="https://route53.amazonaws.com/doc/2013-04-01/"><ChangeInfo><Id>/change/C04661082OQDZOKVTEGNS</Id><Status>PENDING</Status><SubmittedAt>2023-02-09T16:21:52.963Z</SubmittedAt><Comment>create test records</Comment></ChangeInfo></ChangeResourceRecordSetsResponse>'
2023-02-09 19:21:53,064 botocore.hooks [DEBUG] Event needs-retry.route-53.ChangeResourceRecordSets: calling handler <botocore.retryhandler.RetryHandler object at 0x7f912141eec0>
2023-02-09 19:21:53,064 botocore.retryhandler [DEBUG] No retry needed.
[{'Action': 'UPSERT', 'ResourceRecordSet': {'Name': 'foo.pytestexample.com', 'Type': 'CNAME', 'TTL': 123, 'ResourceRecords': [{'Value': 'bar.pytestexample.com.'}]}}, {'Action': 'UPSERT', 'ResourceRecordSet': {'Name': 'spam.pytestexample.com', 'Type': 'CNAME', 'TTL': 123, 'ResourceRecords': [{'Value': 'eggs.pytestexample.com.'}]}}]
2023-02-09 19:21:53,064 botocore.regions [DEBUG] Calling endpoint provider with parameters: {'Region': 'eu-central-1', 'UseDualStack': False, 'UseFIPS': False}
2023-02-09 19:21:53,064 botocore.regions [DEBUG] Endpoint provider result: https://route53.amazonaws.com
2023-02-09 19:21:53,064 botocore.regions [DEBUG] Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None"
2023-02-09 19:21:53,064 botocore.regions [DEBUG] Selected auth type "v4" as "v4" with signing context params: {'region': 'us-east-1', 'signing_name': 'route53'}
2023-02-09 19:21:53,064 botocore.hooks [DEBUG] Event before-parameter-build.route-53.ChangeResourceRecordSets: calling handler <function fix_route53_ids at 0x7f91214dd360>
2023-02-09 19:21:53,064 botocore.handlers [DEBUG] HostedZoneId Z0720548IECD1KKM2UJH -> Z0720548IECD1KKM2UJH
2023-02-09 19:21:53,064 botocore.hooks [DEBUG] Event before-parameter-build.route-53.ChangeResourceRecordSets: calling handler <function generate_idempotent_uuid at 0x7f91214dc4c0>
2023-02-09 19:21:53,064 botocore.hooks [DEBUG] Event before-call.route-53.ChangeResourceRecordSets: calling handler <function add_recursion_detection_header at 0x7f91214dc160>
2023-02-09 19:21:53,064 botocore.hooks [DEBUG] Event before-call.route-53.ChangeResourceRecordSets: calling handler <function inject_api_version_header_if_needed at 0x7f91214ddcf0>
2023-02-09 19:21:53,064 botocore.endpoint [DEBUG] Making request for OperationModel(name=ChangeResourceRecordSets) with params: {'url_path': '/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/', 'query_string': {}, 'method': 'POST', 'headers': {'User-Agent': 'Boto3/1.26.64 Python/3.10.9 Linux/6.1.8-arch1-1 Botocore/1.29.64'}, 'body': b'<ChangeResourceRecordSetsRequest xmlns="https://route53.amazonaws.com/doc/2013-04-01/"><ChangeBatch><Comment>update test records</Comment><Changes><Change><Action>UPSERT</Action><ResourceRecordSet><Name>foo.pytestexample.com</Name><Type>CNAME</Type><TTL>123</TTL><ResourceRecords><ResourceRecord><Value>bar.pytestexample.com.</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></Change><Change><Action>UPSERT</Action><ResourceRecordSet><Name>spam.pytestexample.com</Name><Type>CNAME</Type><TTL>123</TTL><ResourceRecords><ResourceRecord><Value>eggs.pytestexample.com.</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></Change></Changes></ChangeBatch></ChangeResourceRecordSetsRequest>', 'url': 'https://route53.amazonaws.com/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/', 'context': {'client_region': 'aws-global', 'client_config': <botocore.config.Config object at 0x7f912141c400>, 'has_streaming_input': False, 'auth_type': 'v4', 'signing': {'region': 'us-east-1', 'signing_name': 'route53'}}}
2023-02-09 19:21:53,064 botocore.hooks [DEBUG] Event request-created.route-53.ChangeResourceRecordSets: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f912141c2b0>>
2023-02-09 19:21:53,064 botocore.hooks [DEBUG] Event choose-signer.route-53.ChangeResourceRecordSets: calling handler <function set_operation_specific_signer at 0x7f91214dc3a0>
2023-02-09 19:21:53,065 botocore.auth [DEBUG] Calculating signature using v4 auth.
2023-02-09 19:21:53,065 botocore.auth [DEBUG] CanonicalRequest:
POST
/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/

host:route53.amazonaws.com
x-amz-date:20230209T162153Z

host;x-amz-date
ca30625663f861c61993b9d8c0cada8ddce405ed1697a2d5ecf110f2f3d62315
2023-02-09 19:21:53,065 botocore.auth [DEBUG] StringToSign:
AWS4-HMAC-SHA256
20230209T162153Z
20230209/us-east-1/route53/aws4_request
976b17515b406911267f3b06c969b25e6f3f2030e5350947c0dce9579886c52a
2023-02-09 19:21:53,065 botocore.auth [DEBUG] Signature:
a811c6d182ca250f4a843f4f4b1859324d997da62cf589eb0717c0eb977e141e
2023-02-09 19:21:53,065 botocore.hooks [DEBUG] Event request-created.route-53.ChangeResourceRecordSets: calling handler <function add_retry_headers at 0x7f91214de3b0>
2023-02-09 19:21:53,065 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://route53.amazonaws.com/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/, headers={'User-Agent': b'Boto3/1.26.64 Python/3.10.9 Linux/6.1.8-arch1-1 Botocore/1.29.64', 'X-Amz-Date': b'20230209T162153Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIADELETED/20230209/us-east-1/route53/aws4_request, SignedHeaders=host;x-amz-date, Signature=a811c6d182ca250f4a843f4f4b1859324d997da62cf589eb0717c0eb977e141e', 'amz-sdk-invocation-id': b'043ca921-4d6e-43cd-9615-15206ec7c287', 'amz-sdk-request': b'attempt=1', 'Content-Length': '707'}>
2023-02-09 19:21:53,065 botocore.httpsession [DEBUG] Certificate path: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/cacert.pem
2023-02-09 19:21:53,414 urllib3.connectionpool [DEBUG] https://route53.amazonaws.com:443 "POST /2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset/ HTTP/1.1" 200 321
2023-02-09 19:21:53,415 botocore.parsers [DEBUG] Response headers: {'x-amzn-RequestId': '6dc02b0d-2bf2-4e74-8e9f-4053f0c5fc1e', 'Content-Type': 'text/xml', 'Content-Length': '321', 'Date': 'Thu, 09 Feb 2023 16:21:52 GMT'}
2023-02-09 19:21:53,415 botocore.parsers [DEBUG] Response body:
b'<?xml version="1.0"?>\n<ChangeResourceRecordSetsResponse xmlns="https://route53.amazonaws.com/doc/2013-04-01/"><ChangeInfo><Id>/change/C0465444328CKQDMKM65W</Id><Status>PENDING</Status><SubmittedAt>2023-02-09T16:21:53.332Z</SubmittedAt><Comment>update test records</Comment></ChangeInfo></ChangeResourceRecordSetsResponse>'
2023-02-09 19:21:53,415 botocore.hooks [DEBUG] Event needs-retry.route-53.ChangeResourceRecordSets: calling handler <botocore.retryhandler.RetryHandler object at 0x7f912141eec0>
2023-02-09 19:21:53,415 botocore.retryhandler [DEBUG] No retry needed.
2023-02-09 19:21:53,415 botocore.regions [DEBUG] Calling endpoint provider with parameters: {'Region': 'eu-central-1', 'UseDualStack': False, 'UseFIPS': False}
2023-02-09 19:21:53,415 botocore.regions [DEBUG] Endpoint provider result: https://route53.amazonaws.com
2023-02-09 19:21:53,415 botocore.regions [DEBUG] Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None"
2023-02-09 19:21:53,415 botocore.regions [DEBUG] Selected auth type "v4" as "v4" with signing context params: {'region': 'us-east-1', 'signing_name': 'route53'}
2023-02-09 19:21:53,415 botocore.hooks [DEBUG] Event before-parameter-build.route-53.ListResourceRecordSets: calling handler <function fix_route53_ids at 0x7f91214dd360>
2023-02-09 19:21:53,415 botocore.handlers [DEBUG] HostedZoneId Z0720548IECD1KKM2UJH -> Z0720548IECD1KKM2UJH
2023-02-09 19:21:53,415 botocore.hooks [DEBUG] Event before-parameter-build.route-53.ListResourceRecordSets: calling handler <function generate_idempotent_uuid at 0x7f91214dc4c0>
2023-02-09 19:21:53,415 botocore.hooks [DEBUG] Event before-call.route-53.ListResourceRecordSets: calling handler <function add_recursion_detection_header at 0x7f91214dc160>
2023-02-09 19:21:53,416 botocore.hooks [DEBUG] Event before-call.route-53.ListResourceRecordSets: calling handler <function inject_api_version_header_if_needed at 0x7f91214ddcf0>
2023-02-09 19:21:53,416 botocore.endpoint [DEBUG] Making request for OperationModel(name=ListResourceRecordSets) with params: {'url_path': '/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset', 'query_string': {}, 'method': 'GET', 'headers': {'User-Agent': 'Boto3/1.26.64 Python/3.10.9 Linux/6.1.8-arch1-1 Botocore/1.29.64'}, 'body': b'', 'url': 'https://route53.amazonaws.com/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset', 'context': {'client_region': 'aws-global', 'client_config': <botocore.config.Config object at 0x7f912141c400>, 'has_streaming_input': False, 'auth_type': 'v4', 'signing': {'region': 'us-east-1', 'signing_name': 'route53'}}}
2023-02-09 19:21:53,416 botocore.hooks [DEBUG] Event request-created.route-53.ListResourceRecordSets: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f912141c2b0>>
2023-02-09 19:21:53,416 botocore.hooks [DEBUG] Event choose-signer.route-53.ListResourceRecordSets: calling handler <function set_operation_specific_signer at 0x7f91214dc3a0>
2023-02-09 19:21:53,416 botocore.auth [DEBUG] Calculating signature using v4 auth.
2023-02-09 19:21:53,416 botocore.auth [DEBUG] CanonicalRequest:
GET
/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset

host:route53.amazonaws.com
x-amz-date:20230209T162153Z

host;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2023-02-09 19:21:53,416 botocore.auth [DEBUG] StringToSign:
AWS4-HMAC-SHA256
20230209T162153Z
20230209/us-east-1/route53/aws4_request
abc8f0142e5f854bfcc9393aa004e3676ec9aaa2ac25a0f96117f7f967e3a652
2023-02-09 19:21:53,416 botocore.auth [DEBUG] Signature:
0d6dfa6bc37cd10fc763cb6a011ed26d3c62e717a15ae3f666fa30de7829e88b
2023-02-09 19:21:53,416 botocore.hooks [DEBUG] Event request-created.route-53.ListResourceRecordSets: calling handler <function add_retry_headers at 0x7f91214de3b0>
2023-02-09 19:21:53,416 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=GET, url=https://route53.amazonaws.com/2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset, headers={'User-Agent': b'Boto3/1.26.64 Python/3.10.9 Linux/6.1.8-arch1-1 Botocore/1.29.64', 'X-Amz-Date': b'20230209T162153Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIADELETED/20230209/us-east-1/route53/aws4_request, SignedHeaders=host;x-amz-date, Signature=0d6dfa6bc37cd10fc763cb6a011ed26d3c62e717a15ae3f666fa30de7829e88b', 'amz-sdk-invocation-id': b'6191b9c4-f0be-4ea3-82c7-78d47e36f79a', 'amz-sdk-request': b'attempt=1'}>
2023-02-09 19:21:53,416 botocore.httpsession [DEBUG] Certificate path: /home/jack.common/repositories/my/r53issue/venv/lib/python3.10/site-packages/botocore/cacert.pem
2023-02-09 19:21:53,801 urllib3.connectionpool [DEBUG] https://route53.amazonaws.com:443 "GET /2013-04-01/hostedzone/Z0720548IECD1KKM2UJH/rrset HTTP/1.1" 200 1337
2023-02-09 19:21:53,802 botocore.parsers [DEBUG] Response headers: {'x-amzn-RequestId': '15b4f65f-078d-4fe4-8de1-e0b62c643cbb', 'Content-Type': 'text/xml', 'Content-Length': '1337', 'Date': 'Thu, 09 Feb 2023 16:21:53 GMT'}
2023-02-09 19:21:53,802 botocore.parsers [DEBUG] Response body:
b'<?xml version="1.0"?>\n<ListResourceRecordSetsResponse xmlns="https://route53.amazonaws.com/doc/2013-04-01/"><ResourceRecordSets><ResourceRecordSet><Name>pytestexample.com.</Name><Type>NS</Type><TTL>172800</TTL><ResourceRecords><ResourceRecord><Value>ns-1504.awsdns-60.org.</Value></ResourceRecord><ResourceRecord><Value>ns-81.awsdns-10.com.</Value></ResourceRecord><ResourceRecord><Value>ns-621.awsdns-13.net.</Value></ResourceRecord><ResourceRecord><Value>ns-1651.awsdns-14.co.uk.</Value></ResourceRecord></ResourceRecords></ResourceRecordSet><ResourceRecordSet><Name>pytestexample.com.</Name><Type>SOA</Type><TTL>900</TTL><ResourceRecords><ResourceRecord><Value>ns-1504.awsdns-60.org. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400</Value></ResourceRecord></ResourceRecords></ResourceRecordSet><ResourceRecordSet><Name>foo.pytestexample.com.</Name><Type>CNAME</Type><TTL>123</TTL><ResourceRecords><ResourceRecord><Value>bar.pytestexample.com</Value></ResourceRecord></ResourceRecords></ResourceRecordSet><ResourceRecordSet><Name>spam.pytestexample.com.</Name><Type>CNAME</Type><TTL>123</TTL><ResourceRecords><ResourceRecord><Value>eggs.pytestexample.com.</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></ResourceRecordSets><IsTruncated>false</IsTruncated><MaxItems>100</MaxItems></ListResourceRecordSetsResponse>'
2023-02-09 19:21:53,802 botocore.hooks [DEBUG] Event needs-retry.route-53.ListResourceRecordSets: calling handler <botocore.retryhandler.RetryHandler object at 0x7f912141eec0>
2023-02-09 19:21:53,802 botocore.retryhandler [DEBUG] No retry needed.

Final state:

{
    "ResponseMetadata": {
        "RequestId": "15b4f65f-078d-4fe4-8de1-e0b62c643cbb",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
            "x-amzn-requestid": "15b4f65f-078d-4fe4-8de1-e0b62c643cbb",
            "content-type": "text/xml",
            "content-length": "1337",
            "date": "Thu, 09 Feb 2023 16:21:53 GMT"
        },
        "RetryAttempts": 0
    },
    "ResourceRecordSets": [
        {
            "Name": "pytestexample.com.",
            "Type": "NS",
            "TTL": 172800,
            "ResourceRecords": [
                {
                    "Value": "ns-1504.awsdns-60.org."
                },
                {
                    "Value": "ns-81.awsdns-10.com."
                },
                {
                    "Value": "ns-621.awsdns-13.net."
                },
                {
                    "Value": "ns-1651.awsdns-14.co.uk."
                }
            ]
        },
        {
            "Name": "pytestexample.com.",
            "Type": "SOA",
            "TTL": 900,
            "ResourceRecords": [
                {
                    "Value": "ns-1504.awsdns-60.org. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400"
                }
            ]
        },
        {
            "Name": "foo.pytestexample.com.",
            "Type": "CNAME",
            "TTL": 123,
            "ResourceRecords": [
                {
                    "Value": "bar.pytestexample.com"
                }
            ]
        },
        {
            "Name": "spam.pytestexample.com.",
            "Type": "CNAME",
            "TTL": 123,
            "ResourceRecords": [
                {
                    "Value": "eggs.pytestexample.com."
                }
            ]
        }
    ],
    "IsTruncated": false,
    "MaxItems": "100"
}
tim-finnigan commented 1 year ago

Checking in - I escalated this issue internally but the team who looked into the issue was unable to reproduce it. Also it does not look like there was any error in the debug logs. I wonder if this was something transient as we haven't heard any other reports of the issue. If this is still something you're running into and can reproduce consistently please let us know and we can revisit the issue.

github-actions[bot] commented 1 year ago

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

joshsleeper commented 1 year ago

@tim-finnigan huh, well that's a bummer that they couldn't repro! my team at $work just had to work around this exact issue in the last couple weeks, so it's very much still present from our perspective.

lemme see if I can work out a simple repro example here soon to demonstrate what we were seeing.

to be clear, the API behaved exactly as expected when changing a field like TTL across multiple records, properly updating all records provided.

the failure case we saw, where it only updated a single record at a time from the batch, was specifically around having multiple existing CNAME records without the trailing . and trying to bulk update them all to add said trailing ., which things like octodns expect as per-spec

anemyte commented 1 year ago

lemme see if I can work out a simple repro example here soon to demonstrate what we were seeing.

There is a script in the original issue https://github.com/boto/boto3/issues/3579 which I used to reproduce the issue and generate that log above.

joshsleeper commented 1 year ago

right you are @anemyte, I hadn't looked at the script from the previous thread that closely, but from your output it clearly results in a "completed" run, without noted errors, while still leaving some records missing the trailing dot.

@tim-finnigan is that not evidence enough to push back on the devs that customers can clearly repro?

tim-finnigan commented 1 year ago

Thanks for following up, I've reached out again to the Route 53 team and will share any updates here. If you have an AWS Support plan you could also create a case for more direct correspondence.

tim-finnigan commented 6 months ago

Thanks all for your patience here. The Route 53 team has started investigating this issue. I will close this issue because they are tracking it on their end but please feel free to track back for updates in the future.

github-actions[bot] commented 6 months ago

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.