aws s3 cp bucket to bucket copy fails when sse-c is set

[jlax47]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [x] I've gone though the User Guide and the API reference
• [x] I've searched for previous similar issues and didn't find any solution

Describe the bug When doing a bucket to bucket copy using aws s3 cp, the copy fails when SSE-C is enabled on the destination, but not the source. aws s3 cp s3://source/file s3://destination --sse-c --sse-c-key 00000000000000000000000000000000

SDK version number aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off

Platform/OS/Hardware/Device What are you running the cli on? MacOS 11.2.3

To Reproduce (observed behavior)

$ aws s3 cp s3://source-bucket/file s3://dest-bucket/ --sse-c --sse-c-key 00000000000000000000000000000000
fatal error: An error occurred (400) when calling the HeadObject operation: Bad Request

Expected behavior A clear and concise description of what you expected to happen. The CLI should have copied a file from the source bucket to the destination bucket (with SSE-C on the copied file).

Logs/output

2021-03-11 17:02:40,878 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64
2021-03-11 17:02:40,878 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', 'cp', 's3://source-bucket/file', 's3://dest-bucket/', '--sse-c', '--sse-c-key', '00000000000000000000000000000000', '--debug']
2021-03-11 17:02:40,908 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_s3 at 0x104b293a0>
2021-03-11 17:02:40,909 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_ddb at 0x10498c040>
2021-03-11 17:02:40,909 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.configure.configure.ConfigureCommand'>>
2021-03-11 17:02:40,909 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x104928790>
2021-03-11 17:02:40,909 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x104931700>
2021-03-11 17:02:40,909 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function alias_opsworks_cm at 0x104b39d30>
2021-03-11 17:02:40,909 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_history_commands at 0x1049d4c10>
2021-03-11 17:02:40,909 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.devcommands.CLIDevCommand'>>
2021-03-11 17:02:40,909 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_waiters at 0x104b31f70>
2021-03-11 17:02:40,910 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/awscli/data/cli.json
2021-03-11 17:02:40,913 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_types at 0x104a94310>
2021-03-11 17:02:40,913 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function no_sign_request at 0x104a94e50>
2021-03-11 17:02:40,913 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_verify_ssl at 0x104a94dc0>
2021-03-11 17:02:40,913 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_read_timeout at 0x104a94f70>
2021-03-11 17:02:40,914 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_connect_timeout at 0x104a94ee0>
2021-03-11 17:02:40,914 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <built-in method update of dict object at 0x104bd2100>
2021-03-11 17:02:40,914 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off
2021-03-11 17:02:40,914 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', 'cp', 's3://source-bucket/file', 's3://dest-bucket/', '--sse-c', '--sse-c-key', '00000000000000000000000000000000', '--debug']
2021-03-11 17:02:40,914 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_timestamp_parser at 0x104b299d0>
2021-03-11 17:02:40,915 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x10464e700>
2021-03-11 17:02:40,915 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_binary_formatter at 0x104b98430>
2021-03-11 17:02:40,915 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function no_pager_handler at 0x10464a8b0>
2021-03-11 17:02:40,915 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x1046b20d0>
2021-03-11 17:02:40,920 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://169.254.169.254/
2021-03-11 17:02:40,931 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set.
2021-03-11 17:02:40,931 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x1049d4af0>
2021-03-11 17:02:40,931 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_json_file_cache at 0x104985ee0>
2021-03-11 17:02:40,931 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3: calling handler <function add_waiters at 0x104b31f70>
2021-03-11 17:02:40,932 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.s3.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,932 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3_cp: calling handler <function add_waiters at 0x104b31f70>
2021-03-11 17:02:40,934 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.paths: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,934 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.dryrun: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,934 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104677400>
2021-03-11 17:02:40,934 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.quiet: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,934 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104677400>
2021-03-11 17:02:40,934 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.acl: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.follow-symlinks: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104677400>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-c: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-c-key: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104677400>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-kms-key-id: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-c-copy-source: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.sse-c-copy-source-key: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.storage-class: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.grants: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.website-redirect: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.content-type: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,935 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.cache-control: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.content-disposition: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.content-encoding: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.content-language: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.expires: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.source-region: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.only-show-errors: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104677400>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.page-size: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.ignore-glacier-warnings: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104677400>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.force-glacier-transfer: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104677400>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.request-payer: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.metadata: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,936 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.copy-props: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,937 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.cp: calling handler <awscli.argprocess.ParamShorthandParser object at 0x104677400>
2021-03-11 17:02:40,937 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.metadata-directive: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,937 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.expected-size: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,937 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.cp.anonymous: calling handler <awscli.paramfile.URIArgumentHandler object at 0x104c0bf40>
2021-03-11 17:02:40,937 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://169.254.169.254/
2021-03-11 17:02:40,938 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): 169.254.169.254:80
2021-03-11 17:02:41,942 - MainThread - botocore.utils - DEBUG - Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/api/token: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 311, in send
    urllib_response = conn.urlopen(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 92, in _send_request
    rval = super(AWSConnection, self)._send_request(
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 120, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 204, in send
    return super(AWSConnection, self).send(str)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPConnection object at 0x104c50310>, 'Connection to 169.254.169.254 timed out. (connect timeout=1)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/utils.py", line 301, in _fetch_metadata_token
    response = self._session.send(request.prepare())
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 344, in send
    raise ConnectTimeoutError(endpoint_url=request.url, error=e)
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
2021-03-11 17:02:41,957 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (2): 169.254.169.254:80
2021-03-11 17:02:42,961 - MainThread - botocore.utils - DEBUG - Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/meta-data/placement/availability-zone/: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/placement/availability-zone/"
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 311, in send
    urllib_response = conn.urlopen(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 92, in _send_request
    rval = super(AWSConnection, self)._send_request(
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 120, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 204, in send
    return super(AWSConnection, self).send(str)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPConnection object at 0x104c42580>, 'Connection to 169.254.169.254 timed out. (connect timeout=1)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/utils.py", line 349, in _get_request
    response = self._session.send(request.prepare())
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 344, in send
    raise ConnectTimeoutError(endpoint_url=request.url, error=e)
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/placement/availability-zone/"
2021-03-11 17:02:42,963 - MainThread - awscli.utils - DEBUG - Max number of attempts exceeded (1) when attempting to retrieve data from metadata service.
2021-03-11 17:02:42,963 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2021-03-11 17:02:42,964 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity
2021-03-11 17:02:42,964 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso
2021-03-11 17:02:42,964 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2021-03-11 17:02:42,965 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2021-03-11 17:02:42,968 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/data/endpoints.json
2021-03-11 17:02:42,975 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x103cb5d30>
2021-03-11 17:02:43,008 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/data/s3/2006-03-01/service-2.json
2021-03-11 17:02:43,019 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/data/s3/2006-03-01/service-2.sdk-extras.json
2021-03-11 17:02:43,022 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x103c62820>
2021-03-11 17:02:43,022 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x103c625e0>
2021-03-11 17:02:43,032 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60)
2021-03-11 17:02:43,033 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (3): 169.254.169.254:80
2021-03-11 17:02:44,038 - MainThread - botocore.utils - DEBUG - Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/api/token: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 311, in send
    urllib_response = conn.urlopen(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 92, in _send_request
    rval = super(AWSConnection, self)._send_request(
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 120, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 204, in send
    return super(AWSConnection, self).send(str)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPConnection object at 0x1071aa790>, 'Connection to 169.254.169.254 timed out. (connect timeout=1)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/utils.py", line 301, in _fetch_metadata_token
    response = self._session.send(request.prepare())
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 344, in send
    raise ConnectTimeoutError(endpoint_url=request.url, error=e)
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
2021-03-11 17:02:44,041 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (4): 169.254.169.254:80
2021-03-11 17:02:45,044 - MainThread - botocore.utils - DEBUG - Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/meta-data/placement/availability-zone/: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/placement/availability-zone/"
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 311, in send
    urllib_response = conn.urlopen(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 92, in _send_request
    rval = super(AWSConnection, self)._send_request(
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 120, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 204, in send
    return super(AWSConnection, self).send(str)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPConnection object at 0x1071aad00>, 'Connection to 169.254.169.254 timed out. (connect timeout=1)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/utils.py", line 349, in _get_request
    response = self._session.send(request.prepare())
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 344, in send
    raise ConnectTimeoutError(endpoint_url=request.url, error=e)
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/placement/availability-zone/"
2021-03-11 17:02:45,045 - MainThread - awscli.utils - DEBUG - Max number of attempts exceeded (1) when attempting to retrieve data from metadata service.
2021-03-11 17:02:45,045 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x103cb5d30>
2021-03-11 17:02:45,047 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x103c62820>
2021-03-11 17:02:45,047 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x103c625e0>
2021-03-11 17:02:45,052 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60)
2021-03-11 17:02:45,054 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (5): 169.254.169.254:80
2021-03-11 17:02:46,056 - MainThread - botocore.utils - DEBUG - Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/api/token: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 311, in send
    urllib_response = conn.urlopen(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 92, in _send_request
    rval = super(AWSConnection, self)._send_request(
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 120, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 204, in send
    return super(AWSConnection, self).send(str)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPConnection object at 0x107209700>, 'Connection to 169.254.169.254 timed out. (connect timeout=1)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/utils.py", line 301, in _fetch_metadata_token
    response = self._session.send(request.prepare())
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 344, in send
    raise ConnectTimeoutError(endpoint_url=request.url, error=e)
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
2021-03-11 17:02:46,057 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (6): 169.254.169.254:80
2021-03-11 17:02:46,058 - MainThread - botocore.utils - DEBUG - Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/meta-data/placement/availability-zone/: Could not connect to the endpoint URL: "http://169.254.169.254/latest/meta-data/placement/availability-zone/"
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
OSError: [Errno 65] No route to host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 311, in send
    urllib_response = conn.urlopen(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 92, in _send_request
    rval = super(AWSConnection, self)._send_request(
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 120, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/awsrequest.py", line 204, in send
    return super(AWSConnection, self).send(str)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPConnection object at 0x107209c70>: Failed to establish a new connection: [Errno 65] No route to host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/utils.py", line 349, in _get_request
    response = self._session.send(request.prepare())
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/httpsession.py", line 340, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://169.254.169.254/latest/meta-data/placement/availability-zone/"
2021-03-11 17:02:46,058 - MainThread - awscli.utils - DEBUG - Max number of attempts exceeded (1) when attempting to retrieve data from metadata service.
2021-03-11 17:02:46,058 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x103cb5d30>
2021-03-11 17:02:46,059 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x103c62820>
2021-03-11 17:02:46,059 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x103c625e0>
2021-03-11 17:02:46,063 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60)
2021-03-11 17:02:46,064 - MainThread - awscli.customizations.s3.s3handler - DEBUG - Using a multipart threshold of 8388608 and a part size of 8388608
2021-03-11 17:02:46,065 - MainThread - botocore.hooks - DEBUG - Event provide-client-params.s3.HeadObject: calling handler <function base64_decode_input_blobs at 0x104b98b80>
2021-03-11 17:02:46,065 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadObject: calling handler <function sse_md5 at 0x103cda040>
2021-03-11 17:02:46,065 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadObject: calling handler <function validate_bucket_name at 0x103cd5f70>
2021-03-11 17:02:46,065 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x1072618e0>>
2021-03-11 17:02:46,065 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadObject: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x107261940>>
2021-03-11 17:02:46,065 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadObject: calling handler <function generate_idempotent_uuid at 0x103cd5dc0>
2021-03-11 17:02:46,065 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x103cda310>
2021-03-11 17:02:46,066 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x1072618e0>>
2021-03-11 17:02:46,066 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadObject: calling handler <function inject_api_version_header_if_needed at 0x103cdb670>
2021-03-11 17:02:46,066 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=HeadObject) with params: {'url_path': '/source-bucket/file', 'query_string': {}, 'method': 'HEAD', 'headers': {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA=', 'x-amz-server-side-encryption-customer-key-MD5': 'zZ5FnqcIqUjVwvWmyog4zw==', 'User-Agent': 'aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off command/s3.cp'}, 'body': b'', 'url': 'https://s3.amazonaws.com/source-bucket/file', 'context': {'client_region': 'aws-global', 'client_config': <botocore.config.Config object at 0x107224e20>, 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'source-bucket'}}}
2021-03-11 17:02:46,066 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x107224df0>>
2021-03-11 17:02:46,066 - MainThread - botocore.hooks - DEBUG - Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x103cd5ca0>
2021-03-11 17:02:46,066 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadObject: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x107261a00>>
2021-03-11 17:02:46,066 - MainThread - botocore.utils - DEBUG - Defaulting to S3 virtual host style addressing with path style addressing fallback.
2021-03-11 17:02:46,066 - MainThread - botocore.utils - DEBUG - Checking for DNS compatible bucket for: https://s3.amazonaws.com/source-bucket/file
2021-03-11 17:02:46,066 - MainThread - botocore.utils - DEBUG - URI updated to: https://source-bucket.s3.amazonaws.com/file
2021-03-11 17:02:46,067 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2021-03-11 17:02:46,067 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
HEAD
/file

host:source-bucket.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20210311T220246Z
x-amz-server-side-encryption-customer-algorithm:AES256
x-amz-server-side-encryption-customer-key:MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA=
x-amz-server-side-encryption-customer-key-md5:zZ5FnqcIqUjVwvWmyog4zw==

host;x-amz-content-sha256;x-amz-date;x-amz-server-side-encryption-customer-algorithm;x-amz-server-side-encryption-customer-key;x-amz-server-side-encryption-customer-key-md5
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2021-03-11 17:02:46,067 - MainThread - botocore.auth - DEBUG - StringToSign: [Redacted]
2021-03-11 17:02:46,067 - MainThread - botocore.auth - DEBUG - Signature: [Redacted]
2021-03-11 17:02:46,067 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://source-bucket.s3.amazonaws.com/file, headers={'x-amz-server-side-encryption-customer-algorithm': b'AES256', 'x-amz-server-side-encryption-customer-key': b'MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA=', 'x-amz-server-side-encryption-customer-key-MD5': b'zZ5FnqcIqUjVwvWmyog4zw==', 'User-Agent': b'aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off command/s3.cp', 'X-Amz-Date': b'20210311T220246Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': [redacted]}>
2021-03-11 17:02:46,067 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/cacert.pem
2021-03-11 17:02:46,068 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): source-bucket.s3.amazonaws.com:443
2021-03-11 17:02:46,346 - MainThread - urllib3.connectionpool - DEBUG - https://source-bucket.s3.amazonaws.com:443 "HEAD /file HTTP/1.1" 400 0
2021-03-11 17:02:46,347 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-request-id': '91J2KD2ASEFGCPZZ', 'x-amz-id-2': 'ZSEPz5Hhr318EZJVmeOBRhaRCEwxENMaVe0D0e2hvMERiC3hPMXJUw3pvl9jQsXirGx5eYkEyKY=', 'Content-Type': 'application/xml', 'Date': 'Thu, 11 Mar 2021 22:02:45 GMT', 'Server': 'AmazonS3', 'Connection': 'close'}
2021-03-11 17:02:46,347 - MainThread - botocore.parsers - DEBUG - Response body:
b''
2021-03-11 17:02:46,350 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadObject: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x1072618b0>>
2021-03-11 17:02:46,350 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2021-03-11 17:02:46,350 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x1072618e0>>
2021-03-11 17:02:46,350 - MainThread - botocore.hooks - DEBUG - Event provide-client-params.s3.HeadBucket: calling handler <function base64_decode_input_blobs at 0x104b98b80>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <function validate_bucket_name at 0x103cd5f70>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x1072618e0>>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x107261940>>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.HeadBucket: calling handler <function generate_idempotent_uuid at 0x103cd5dc0>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadBucket: calling handler <function add_expect_header at 0x103cda310>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadBucket: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x1072618e0>>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.HeadBucket: calling handler <function inject_api_version_header_if_needed at 0x103cdb670>
2021-03-11 17:02:46,351 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=HeadBucket) with params: {'url_path': '/source-bucket', 'query_string': {}, 'method': 'HEAD', 'headers': {'User-Agent': 'aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off command/s3.cp'}, 'body': b'', 'url': 'https://s3.amazonaws.com/source-bucket', 'context': {'client_region': 'aws-global', 'client_config': <botocore.config.Config object at 0x107224e20>, 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'source-bucket'}}}
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.HeadBucket: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x107224df0>>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event choose-signer.s3.HeadBucket: calling handler <function set_operation_specific_signer at 0x103cd5ca0>
2021-03-11 17:02:46,351 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadBucket: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x107261a00>>
2021-03-11 17:02:46,351 - MainThread - botocore.utils - DEBUG - Checking for DNS compatible bucket for: https://s3.amazonaws.com/source-bucket
2021-03-11 17:02:46,352 - MainThread - botocore.utils - DEBUG - URI updated to: https://source-bucket.s3.amazonaws.com/
2021-03-11 17:02:46,352 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2021-03-11 17:02:46,352 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
HEAD
/

host:source-bucket.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20210311T220246Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2021-03-11 17:02:46,352 - MainThread - botocore.auth - DEBUG - StringToSign: [Redacted]
2021-03-11 17:02:46,352 - MainThread - botocore.auth - DEBUG - Signature: [Redacted]
2021-03-11 17:02:46,352 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://source-bucket.s3.amazonaws.com/, headers={'User-Agent': b'aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off command/s3.cp', 'X-Amz-Date': b'20210311T220246Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': 'Authorization': [redacted]}>
2021-03-11 17:02:46,352 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/cacert.pem
2021-03-11 17:02:46,352 - MainThread - urllib3.connectionpool - DEBUG - Resetting dropped connection: source-bucket.s3.amazonaws.com
2021-03-11 17:02:46,628 - MainThread - urllib3.connectionpool - DEBUG - https://source-bucket.s3.amazonaws.com:443 "HEAD / HTTP/1.1" 400 0
2021-03-11 17:02:46,629 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-bucket-region': 'us-west-1', 'x-amz-request-id': '91J0VSX7ZVHZXAJN', 'x-amz-id-2': 'txuXBcAACUnbSUzx4lLUzfT14y9bArMnLJ2amQcV3aFJyFtZj5mt/CXuQQsEA8/qTYRV+ORMIXQ=', 'Content-Type': 'application/xml', 'Date': 'Thu, 11 Mar 2021 22:02:45 GMT', 'Server': 'AmazonS3', 'Connection': 'close'}
2021-03-11 17:02:46,629 - MainThread - botocore.parsers - DEBUG - Response body:
b''
2021-03-11 17:02:46,630 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x1072618b0>>
2021-03-11 17:02:46,630 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2021-03-11 17:02:46,630 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x1072618e0>>
2021-03-11 17:02:46,630 - MainThread - botocore.utils - DEBUG - S3 client configured for region aws-global but the bucket source-bucket is in region us-west-1; Please configure the proper region to avoid multiple unnecessary redirects and signing attempts.
2021-03-11 17:02:46,630 - MainThread - botocore.utils - DEBUG - Updating URI from https://s3.amazonaws.com/source-bucket to https://s3.us-west-1.amazonaws.com/source-bucket
2021-03-11 17:02:46,630 - MainThread - botocore.endpoint - DEBUG - Response received to retry, sleeping for 0 seconds
2021-03-11 17:02:46,630 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.HeadBucket: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x107224df0>>
2021-03-11 17:02:46,630 - MainThread - botocore.hooks - DEBUG - Event choose-signer.s3.HeadBucket: calling handler <function set_operation_specific_signer at 0x103cd5ca0>
2021-03-11 17:02:46,630 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadBucket: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x107261a00>>
2021-03-11 17:02:46,630 - MainThread - botocore.utils - DEBUG - Checking for DNS compatible bucket for: https://s3.us-west-1.amazonaws.com/source-bucket
2021-03-11 17:02:46,630 - MainThread - botocore.utils - DEBUG - URI updated to: https://source-bucket.s3.us-west-1.amazonaws.com/
2021-03-11 17:02:46,631 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2021-03-11 17:02:46,631 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
HEAD
/

host:source-bucket.s3.us-west-1.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20210311T220246Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2021-03-11 17:02:46,631 - MainThread - botocore.auth - DEBUG - StringToSign:[Redacted]
2021-03-11 17:02:46,631 - MainThread - botocore.auth - DEBUG - Signature: [Redacted]
2021-03-11 17:02:46,631 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://source-bucket.s3.us-west-1.amazonaws.com/, headers={'User-Agent': b'aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off command/s3.cp', 'X-Amz-Date': b'20210311T220246Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': 'Authorization': [redacted]}>
2021-03-11 17:02:46,631 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/cacert.pem
2021-03-11 17:02:46,631 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): source-bucket.s3.us-west-1.amazonaws.com:443
2021-03-11 17:02:46,933 - MainThread - urllib3.connectionpool - DEBUG - https://source-bucket.s3.us-west-1.amazonaws.com:443 "HEAD / HTTP/1.1" 200 0
2021-03-11 17:02:46,933 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-id-2': 'E4Krs1NMLLx3rIO8c5ZXNvTsHNIB09Va4tK4G7ldzFV/DgMeYOeVyzRNO9BiraqrI6rVNwYKciE=', 'x-amz-request-id': '91J9EF5ZDSA349R2', 'Date': 'Thu, 11 Mar 2021 22:02:47 GMT', 'x-amz-bucket-region': 'us-west-1', 'Content-Type': 'application/xml', 'Server': 'AmazonS3'}
2021-03-11 17:02:46,933 - MainThread - botocore.parsers - DEBUG - Response body:
b''
2021-03-11 17:02:46,933 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x1072618b0>>
2021-03-11 17:02:46,933 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2021-03-11 17:02:46,933 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadBucket: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x1072618e0>>
2021-03-11 17:02:46,933 - MainThread - botocore.utils - DEBUG - S3 request was previously redirected, not redirecting.
2021-03-11 17:02:46,933 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.HeadBucket: calling handler <function enhance_error_msg at 0x104b295e0>
2021-03-11 17:02:46,933 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.HeadBucket: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x107261430>>
2021-03-11 17:02:46,933 - MainThread - botocore.utils - DEBUG - S3 client configured for region aws-global but the bucket source-bucket is in region us-west-1; Please configure the proper region to avoid multiple unnecessary redirects and signing attempts.
2021-03-11 17:02:46,934 - MainThread - botocore.utils - DEBUG - Updating URI from https://s3.amazonaws.com/source-bucket/file to https://s3.us-west-1.amazonaws.com/source-bucket/file
2021-03-11 17:02:46,934 - MainThread - botocore.endpoint - DEBUG - Response received to retry, sleeping for 0 seconds
2021-03-11 17:02:46,934 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x107224df0>>
2021-03-11 17:02:46,934 - MainThread - botocore.hooks - DEBUG - Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x103cd5ca0>
2021-03-11 17:02:46,934 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.HeadObject: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x107261a00>>
2021-03-11 17:02:46,934 - MainThread - botocore.utils - DEBUG - Checking for DNS compatible bucket for: https://s3.us-west-1.amazonaws.com/source-bucket/file
2021-03-11 17:02:46,934 - MainThread - botocore.utils - DEBUG - URI updated to: https://source-bucket.s3.us-west-1.amazonaws.com/file
2021-03-11 17:02:46,934 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2021-03-11 17:02:46,934 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
HEAD
/file

host:source-bucket.s3.us-west-1.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20210311T220246Z
x-amz-server-side-encryption-customer-algorithm:AES256
x-amz-server-side-encryption-customer-key:MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA=
x-amz-server-side-encryption-customer-key-md5:zZ5FnqcIqUjVwvWmyog4zw==

host;x-amz-content-sha256;x-amz-date;x-amz-server-side-encryption-customer-algorithm;x-amz-server-side-encryption-customer-key;x-amz-server-side-encryption-customer-key-md5
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2021-03-11 17:02:46,934 - MainThread - botocore.auth - DEBUG - StringToSign: [Redacted]
2021-03-11 17:02:46,934 - MainThread - botocore.auth - DEBUG - Signature: [Redacted]
2021-03-11 17:02:46,934 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://source-bucket.s3.us-west-1.amazonaws.com/file, headers={'x-amz-server-side-encryption-customer-algorithm': b'AES256', 'x-amz-server-side-encryption-customer-key': b'MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA=', 'x-amz-server-side-encryption-customer-key-MD5': b'zZ5FnqcIqUjVwvWmyog4zw==', 'User-Agent': b'aws-cli/2.1.29 Python/3.9.2 Darwin/20.3.0 source/x86_64 prompt/off command/s3.cp', 'X-Amz-Date': b'20210311T220246Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': 'Authorization': [redacted]}>
2021-03-11 17:02:46,935 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/cacert.pem
2021-03-11 17:02:47,008 - MainThread - urllib3.connectionpool - DEBUG - https://source-bucket.s3.us-west-1.amazonaws.com:443 "HEAD /file HTTP/1.1" 400 0
2021-03-11 17:02:47,009 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-request-id': '91JCXW41V4N07XYQ', 'x-amz-id-2': 'tfeoePtbg9ElkrtN4gw80I1l0Zjyq4ARdlEdDS+P+IqbJTqlgDT18FPQr/sv9/PTF7K0gKCAHYQ=', 'Content-Type': 'application/xml', 'Date': 'Thu, 11 Mar 2021 22:02:46 GMT', 'Server': 'AmazonS3', 'Connection': 'close'}
2021-03-11 17:02:47,009 - MainThread - botocore.parsers - DEBUG - Response body:
b''
2021-03-11 17:02:47,009 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadObject: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x1072618b0>>
2021-03-11 17:02:47,009 - MainThread - botocore.retries.standard - DEBUG - Not retrying request.
2021-03-11 17:02:47,009 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x1072618e0>>
2021-03-11 17:02:47,009 - MainThread - botocore.utils - DEBUG - S3 request was previously redirected, not redirecting.
2021-03-11 17:02:47,009 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.HeadObject: calling handler <function enhance_error_msg at 0x104b295e0>
2021-03-11 17:02:47,009 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.HeadObject: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x107261430>>
2021-03-11 17:02:47,010 - MainThread - awscli.customizations.s3.results - DEBUG - Exception caught during command execution: An error occurred (400) when calling the HeadObject operation: Bad Request
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/awscli/customizations/s3/s3handler.py", line 169, in call
    for fileinfo in fileinfos:
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/awscli/customizations/s3/fileinfobuilder.py", line 31, in call
    for file_base in files:
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/awscli/customizations/s3/filegenerator.py", line 142, in call
    for src_path, extra_information in file_iterator:
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/awscli/customizations/s3/filegenerator.py", line 318, in list_objects
    yield self._list_single_object(s3_path)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/awscli/customizations/s3/filegenerator.py", line 355, in _list_single_object
    response = self._client.head_object(**params)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/client.py", line 249, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/Cellar/awscli/2.1.29/libexec/lib/python3.9/site-packages/botocore/client.py", line 568, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (400) when calling the HeadObject operation: Bad Request
fatal error: An error occurred (400) when calling the HeadObject operation: Bad Request
2021-03-11 17:02:47,018 - Thread-1 - awscli.customizations.s3.results - DEBUG - Shutdown request received in result processing thread, shutting down result thread.

Additional context The problem stems from the fact that the cli is sending a head request to the source bucket with the SSE-C headers set, when it should not.

[stobrien89]

Hi @jlax47,

Thanks for the report! I can reproduce. Marking this as a bug.

[stobrien89]

Following up after some additional testing— confirmed the S3 API supports encrypting a previously unencrypted object being transferred from one bucket to another. This works:

aws s3api copy-object --copy-source sourcebucket/index.html --key index.html --bucket destinationbucket --sse-customer-algorithm AES256 --sse-customer-key somekey

[DailyDreaming]

I'm also affected by this issue. Thanks in advance!

[Da-1kun]

May I work on it?

[stobrien89]

Hi @Da-1kun,

You're welcome to work on it! We'll need accompanying tests for any pull requests submitted for this fix.

[Da-1kun]

@stobrien89 Is there any document which describes how to run the AWS CLI test? → I found out how to run the tests.

[Da-1kun]

@stobrien89 @kdaily @jlax47 I made a PR, could you check it ?

[KarlKl]

Any update on this? It's been a year since this bug was reported and the only PR was closed.. 😬 The workaround with the s3api is unfortunately not applicable for my case, because I want to copy a folder recursively and that does not work with Copy-Object as far as I have seen. 😞