terricain / aioboto3

Wrapper to use boto3 resources with the aiobotocore async backend
Apache License 2.0
748 stars 76 forks source link

Error when accessing Server Side Encrypted objects in s3. #240

Closed athul-mindtickle closed 2 years ago

athul-mindtickle commented 3 years ago
* Python version: 3.7.8
* Operating System: Mac OSX (11.5.2) Big Sur

### Description

I am trying to upload a file object to a bucket with Server Side Encryption at rest with AWS Managed keys. 

1. Upload a file using `s3.upload_fileobj` as mentioned in the [documentation](https://aioboto3.readthedocs.io/en/latest/usage.html#upload).

      The code snippet is given below

      ```python
      file = 'test.csv'
      session = aioboto3.Session()
      async with session.client("s3", endpoint_url=TestConfig.S3_ENDPOINT_URL, config=config_, region_name='eu-west-2') as s3:
          with open(file, 'rb') as spfp:
              print(f"Uploading {file} to s3")
              await s3.upload_fileobj(spfp, TestConfig.S3_BUCKET, 'test.csv')
              print(f"Finished Uploading {file} to s3")
When I run the code I get the error below looking for SSL Certificate. 
<details>
<summary>**Click to expand Error Traceback**</summary>
    ```python

    /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py in create_connection(self, protocol_factory, host, port, ssl, family, proto, flags, sock, local_addr, server_hostname, ssl_handshake_timeout)
        988             sock, protocol_factory, ssl, server_hostname,
    --> 989             ssl_handshake_timeout=ssl_handshake_timeout)
        990         if self._debug:

    /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py in _create_connection_transport(self, sock, protocol_factory, ssl, server_hostname, server_side, ssl_handshake_timeout)
       1016         try:
    -> 1017             await waiter
       1018         except:

    /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py in data_received(self, data)
        529         try:
    --> 530             ssldata, appdata = self._sslpipe.feed_ssldata(data)
        531         except Exception as e:

    /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py in feed_ssldata(self, data, only_handshake)
        188                 # Call do_handshake() until it doesn't raise anymore.
    --> 189                 self._sslobj.do_handshake()
        190                 self._state = _WRAPPED

    /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py in do_handshake(self)
        773         """Start the SSL/TLS handshake."""
    --> 774         self._sslobj.do_handshake()
        775 

    SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)

    The above exception was the direct cause of the following exception:

    ClientConnectorCertificateError           Traceback (most recent call last)
    /var/folders/xc/rldtw_ms7d3czvzl5bv02h340000gn/T/ipykernel_4248/1764226881.py in async-def-wrapper()
          7         print(f"Finished Uploading {file} to s3")
          8 

     ~/**/**/*********/venv/lib/python3.7/site-packages/aioboto3/s3/inject.py in upload_fileobj(self, Fileobj, Bucket, Key, ExtraArgs, Callback, Config, Processing)
        185 
        186     # Start multipart upload
    --> 187     resp = await self.create_multipart_upload(Bucket=Bucket, Key=Key, **kwargs)
        188     upload_id = resp['UploadId']
        189     finished_parts = []

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/client.py in _make_api_call(self, operation_name, api_params)
        140         else:
        141             http, parsed_response = await self._make_request(
    --> 142                 operation_model, request_dict, request_context)
        143 
        144         await self.meta.events.emit(

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/client.py in _make_request(self, operation_model, request_dict, request_context)
        159     async def _make_request(self, operation_model, request_dict, request_context):
        160         try:
    --> 161             return await self._endpoint.make_request(operation_model, request_dict)
        162         except Exception as e:
        163             await self.meta.events.emit(

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _send_request(self, request_dict, operation_model)
         85     async def _send_request(self, request_dict, operation_model):
         86         attempts = 1
    ---> 87         request = await self.create_request(request_dict, operation_model)
         88         context = request_dict['context']
         89         success_response, exception = await self._get_response(

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in create_request(self, params, operation_model)
         79                 op_name=operation_model.name)
         80             await self._event_emitter.emit(event_name, request=request,
    ---> 81                                            operation_name=operation_model.name)
         82         prepared_request = self.prepare_request(request)
         83         return prepared_request

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/hooks.py in _emit(self, event_name, kwargs, stop_on_response)
         25             # Await the handler if its a coroutine.
         26             if asyncio.iscoroutinefunction(handler):
    ---> 27                 response = await handler(**kwargs)
         28             else:
         29                 response = handler(**kwargs)

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/signers.py in handler(self, operation_name, request, **kwargs)
         14         # this method is invoked to sign the request.
         15         # Don't call this method directly.
    ---> 16         return await self.sign(operation_name, request)
         17 
         18     async def sign(self, operation_name, request, region_name=None,

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/signers.py in sign(self, operation_name, request, region_name, signing_type, expires_in, signing_name)
         53                 kwargs['signing_name'] = signing_context['signing_name']
         54             try:
    ---> 55                 auth = await self.get_auth_instance(**kwargs)
         56             except UnknownSignatureVersionError as e:
         57                 if signing_type != 'standard':

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/signers.py in get_auth_instance(self, signing_name, region_name, signature_version, **kwargs)
         75         frozen_credentials = None
         76         if self._credentials is not None:
    ---> 77             frozen_credentials = await self._credentials.get_frozen_credentials()
         78         kwargs['credentials'] = frozen_credentials
         79         if cls.REQUIRES_REGION:

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in get_frozen_credentials(self)
        314 
        315     async def get_frozen_credentials(self):
    --> 316         await self._refresh()
        317         return self._frozen_credentials
        318 

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in _refresh(self)
        276                 is_mandatory_refresh = self.refresh_needed(
        277                     self._mandatory_refresh_timeout)
    --> 278                 await self._protected_refresh(is_mandatory=is_mandatory_refresh)
        279                 return
        280         elif self.refresh_needed(self._mandatory_refresh_timeout):

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in _protected_refresh(self, is_mandatory)
        289     async def _protected_refresh(self, is_mandatory):
        290         try:
    --> 291             metadata = await self._refresh_using()
        292         except Exception:
        293             period_name = 'mandatory' if is_mandatory else 'advisory'

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in fetch_credentials(self)
        343 
        344     async def fetch_credentials(self):
    --> 345         return await self._get_cached_credentials()
        346 
        347     async def _get_cached_credentials(self):

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in _get_cached_credentials(self)
        353         response = self._load_from_cache()
        354         if response is None:
    --> 355             response = await self._get_credentials()
        356             self._write_to_cache(response)
        357         else:

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in _get_credentials(self)
        870             }
        871             try:
    --> 872                 response = await client.get_role_credentials(**kwargs)
        873             except client.exceptions.UnauthorizedException:
        874                 raise UnauthorizedSSOTokenError()

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/client.py in _make_api_call(self, operation_name, api_params)
        140         else:
        141             http, parsed_response = await self._make_request(
    --> 142                 operation_model, request_dict, request_context)
        143 
        144         await self.meta.events.emit(

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/client.py in _make_request(self, operation_model, request_dict, request_context)
        159     async def _make_request(self, operation_model, request_dict, request_context):
        160         try:
    --> 161             return await self._endpoint.make_request(operation_model, request_dict)
        162         except Exception as e:
        163             await self.meta.events.emit(

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _send_request(self, request_dict, operation_model)
         91         while await self._needs_retry(attempts, operation_model,
         92                                       request_dict, success_response,
    ---> 93                                       exception):
         94             attempts += 1
         95             # If there is a stream associated with the request, we need

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _needs_retry(self, attempts, operation_model, request_dict, response, caught_exception)
        225             event_name, response=response, endpoint=self,
        226             operation=operation_model, attempts=attempts,
    --> 227             caught_exception=caught_exception, request_dict=request_dict)
        228         handler_response = first_non_none_response(responses)
        229         if handler_response is None:

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/hooks.py in _emit(self, event_name, kwargs, stop_on_response)
         27                 response = await handler(**kwargs)
         28             else:
    ---> 29                 response = handler(**kwargs)
         30 
         31             responses.append((handler, response))

     ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in __call__(self, attempts, response, caught_exception, **kwargs)
        181 
        182         """
    --> 183         if self._checker(attempts, response, caught_exception):
        184             result = self._action(attempts=attempts)
        185             logger.debug("Retry needed, action of: %s", result)

     ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception)
        249     def __call__(self, attempt_number, response, caught_exception):
        250         should_retry = self._should_retry(attempt_number, response,
    --> 251                                           caught_exception)
        252         if should_retry:
        253             if attempt_number >= self._max_attempts:

     ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in _should_retry(self, attempt_number, response, caught_exception)
        275             # If we've exceeded the max attempts we just let the exception
        276             # propogate if one has occurred.
    --> 277             return self._checker(attempt_number, response, caught_exception)
        278 
        279 

     ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception)
        315         for checker in self._checkers:
        316             checker_response = checker(attempt_number, response,
    --> 317                                        caught_exception)
        318             if checker_response:
        319                 return checker_response

     ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception)
        221         elif caught_exception is not None:
        222             return self._check_caught_exception(
    --> 223                 attempt_number, caught_exception)
        224         else:
        225             raise ValueError("Both response and caught_exception are None.")

     ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in _check_caught_exception(self, attempt_number, caught_exception)
        357         # the MaxAttemptsDecorator is not interested in retrying the exception
        358         # then this exception just propogates out past the retry code.
    --> 359         raise caught_exception

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _do_get_response(self, request, operation_model)
        156             http_response = first_non_none_response(responses)
        157             if http_response is None:
    --> 158                 http_response = await self._send(request)
        159         except aiohttp.ClientConnectionError as e:
        160             e.request = request  # botocore expects the request property

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _send(self, request)
        267         url = URL(url, encoded=True)
        268         resp = await self.http_session.request(
    --> 269             request.method, url=url, headers=headers_, data=data, proxy=proxy)
        270 
        271         # If we're not streaming, read the content so we can retry any timeout

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/client.py in _request(self, method, str_or_url, params, data, json, cookies, headers, skip_auto_headers, auth, allow_redirects, max_redirects, compress, chunked, expect100, raise_for_status, read_until_eof, proxy, proxy_auth, timeout, verify_ssl, fingerprint, ssl_context, ssl, proxy_headers, trace_request_ctx)
        474                                 req,
        475                                 traces=traces,
    --> 476                                 timeout=real_timeout
        477                             )
        478                     except asyncio.TimeoutError as exc:

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in connect(self, req, traces, timeout)
        520 
        521             try:
    --> 522                 proto = await self._create_connection(req, traces, timeout)
        523                 if self._closed:
        524                     proto.close()

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _create_connection(self, req, traces, timeout)
        852         else:
        853             _, proto = await self._create_direct_connection(
    --> 854                 req, traces, timeout)
        855 
        856         return proto

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _create_direct_connection(self, req, traces, timeout, client_error)
        990         else:
        991             assert last_exc is not None
    --> 992             raise last_exc
        993 
        994     async def _create_proxy_connection(

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _create_direct_connection(self, req, traces, timeout, client_error)
        972                     server_hostname=hinfo['hostname'] if sslcontext else None,
        973                     local_addr=self._local_addr,
    --> 974                     req=req, client_error=client_error)
        975             except ClientConnectorError as exc:
        976                 last_exc = exc

     ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _wrap_create_connection(self, req, timeout, client_error, *args, **kwargs)
        925         except cert_errors as exc:
        926             raise ClientConnectorCertificateError(
    --> 927                 req.connection_key, exc) from exc
        928         except ssl_errors as exc:
        929             raise ClientConnectorSSLError(req.connection_key, exc) from exc

    ClientConnectorCertificateError: Cannot connect to host portal.sso.ap-southeast-1.amazonaws.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')]

    ```
 </details>
  1. Get the presigned URL. The code snippet is given below.

    session = aioboto3.Session()
    async with session.client("s3", endpoint_url=TestConfig.S3_ENDPOINT_URL, config=config_, region_name='eu-west-2') as s3:
    
        await s3.generate_presigned_url(
            ClientMethod='get_object',
               Params = {
                'Bucket': TestConfig.S3_MEDIA_BUCKET,
                'Key': 'encrypt-key.txt',
            },
        ExpiresIn=1000,
        )

    When I run the code I get the error below looking for SSL Certificate, again!

    **Click to expand Error Traceback**

    ```python --------------------------------------------------------------------------- SSLCertVerificationError Traceback (most recent call last) ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _wrap_create_connection(self, req, timeout, client_error, *args, **kwargs) 923 Tuple[asyncio.Transport, ResponseHandler], --> 924 await self._loop.create_connection(*args, **kwargs)) 925 except cert_errors as exc: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py in create_connection(self, protocol_factory, host, port, ssl, family, proto, flags, sock, local_addr, server_hostname, ssl_handshake_timeout) 988 sock, protocol_factory, ssl, server_hostname, --> 989 ssl_handshake_timeout=ssl_handshake_timeout) 990 if self._debug: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py in _create_connection_transport(self, sock, protocol_factory, ssl, server_hostname, server_side, ssl_handshake_timeout) 1016 try: -> 1017 await waiter 1018 except: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py in data_received(self, data) 529 try: --> 530 ssldata, appdata = self._sslpipe.feed_ssldata(data) 531 except Exception as e: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py in feed_ssldata(self, data, only_handshake) 188 # Call do_handshake() until it doesn't raise anymore. --> 189 self._sslobj.do_handshake() 190 self._state = _WRAPPED /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py in do_handshake(self) 773 """Start the SSL/TLS handshake.""" --> 774 self._sslobj.do_handshake() 775 SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091) The above exception was the direct cause of the following exception: ClientConnectorCertificateError Traceback (most recent call last) /var/folders/xc/rldtw_ms7d3czvzl5bv02h340000gn/T/ipykernel_4248/1386305885.py in async-def-wrapper() 12 ) 13 ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/signers.py in generate_presigned_url(self, ClientMethod, Params, ExpiresIn, HttpMethod) 246 return await request_signer.generate_presigned_url( 247 request_dict=request_dict, expires_in=expires_in, --> 248 operation_name=operation_name) 249 250 ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/signers.py in generate_presigned_url(self, request_dict, operation_name, expires_in, region_name, signing_name) 121 request = create_request_object(request_dict) 122 await self.sign(operation_name, request, region_name, --> 123 'presign-url', expires_in, signing_name) 124 125 request.prepare() ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/signers.py in sign(self, operation_name, request, region_name, signing_type, expires_in, signing_name) 53 kwargs['signing_name'] = signing_context['signing_name'] 54 try: ---> 55 auth = await self.get_auth_instance(**kwargs) 56 except UnknownSignatureVersionError as e: 57 if signing_type != 'standard': ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/signers.py in get_auth_instance(self, signing_name, region_name, signature_version, **kwargs) 75 frozen_credentials = None 76 if self._credentials is not None: ---> 77 frozen_credentials = await self._credentials.get_frozen_credentials() 78 kwargs['credentials'] = frozen_credentials 79 if cls.REQUIRES_REGION: ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in get_frozen_credentials(self) 314 315 async def get_frozen_credentials(self): --> 316 await self._refresh() 317 return self._frozen_credentials 318 ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in _refresh(self) 276 is_mandatory_refresh = self.refresh_needed( 277 self._mandatory_refresh_timeout) --> 278 await self._protected_refresh(is_mandatory=is_mandatory_refresh) 279 return 280 elif self.refresh_needed(self._mandatory_refresh_timeout): ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in _protected_refresh(self, is_mandatory) 289 async def _protected_refresh(self, is_mandatory): 290 try: --> 291 metadata = await self._refresh_using() 292 except Exception: 293 period_name = 'mandatory' if is_mandatory else 'advisory' ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in fetch_credentials(self) 343 344 async def fetch_credentials(self): --> 345 return await self._get_cached_credentials() 346 347 async def _get_cached_credentials(self): ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in _get_cached_credentials(self) 353 response = self._load_from_cache() 354 if response is None: --> 355 response = await self._get_credentials() 356 self._write_to_cache(response) 357 else: ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/credentials.py in _get_credentials(self) 870 } 871 try: --> 872 response = await client.get_role_credentials(**kwargs) 873 except client.exceptions.UnauthorizedException: 874 raise UnauthorizedSSOTokenError() ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/client.py in _make_api_call(self, operation_name, api_params) 140 else: 141 http, parsed_response = await self._make_request( --> 142 operation_model, request_dict, request_context) 143 144 await self.meta.events.emit( ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/client.py in _make_request(self, operation_model, request_dict, request_context) 159 async def _make_request(self, operation_model, request_dict, request_context): 160 try: --> 161 return await self._endpoint.make_request(operation_model, request_dict) 162 except Exception as e: 163 await self.meta.events.emit( ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _send_request(self, request_dict, operation_model) 91 while await self._needs_retry(attempts, operation_model, 92 request_dict, success_response, ---> 93 exception): 94 attempts += 1 95 # If there is a stream associated with the request, we need ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _needs_retry(self, attempts, operation_model, request_dict, response, caught_exception) 225 event_name, response=response, endpoint=self, 226 operation=operation_model, attempts=attempts, --> 227 caught_exception=caught_exception, request_dict=request_dict) 228 handler_response = first_non_none_response(responses) 229 if handler_response is None: ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/hooks.py in _emit(self, event_name, kwargs, stop_on_response) 27 response = await handler(**kwargs) 28 else: ---> 29 response = handler(**kwargs) 30 31 responses.append((handler, response)) ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in __call__(self, attempts, response, caught_exception, **kwargs) 181 182 """ --> 183 if self._checker(attempts, response, caught_exception): 184 result = self._action(attempts=attempts) 185 logger.debug("Retry needed, action of: %s", result) ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception) 249 def __call__(self, attempt_number, response, caught_exception): 250 should_retry = self._should_retry(attempt_number, response, --> 251 caught_exception) 252 if should_retry: 253 if attempt_number >= self._max_attempts: ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in _should_retry(self, attempt_number, response, caught_exception) 275 # If we've exceeded the max attempts we just let the exception 276 # propogate if one has occurred. --> 277 return self._checker(attempt_number, response, caught_exception) 278 279 ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception) 315 for checker in self._checkers: 316 checker_response = checker(attempt_number, response, --> 317 caught_exception) 318 if checker_response: 319 return checker_response ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception) 221 elif caught_exception is not None: 222 return self._check_caught_exception( --> 223 attempt_number, caught_exception) 224 else: 225 raise ValueError("Both response and caught_exception are None.") ~/**/**/*********/venv/lib/python3.7/site-packages/botocore/retryhandler.py in _check_caught_exception(self, attempt_number, caught_exception) 357 # the MaxAttemptsDecorator is not interested in retrying the exception 358 # then this exception just propogates out past the retry code. --> 359 raise caught_exception ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _do_get_response(self, request, operation_model) 156 http_response = first_non_none_response(responses) 157 if http_response is None: --> 158 http_response = await self._send(request) 159 except aiohttp.ClientConnectionError as e: 160 e.request = request # botocore expects the request property ~/**/**/*********/venv/lib/python3.7/site-packages/aiobotocore/endpoint.py in _send(self, request) 267 url = URL(url, encoded=True) 268 resp = await self.http_session.request( --> 269 request.method, url=url, headers=headers_, data=data, proxy=proxy) 270 271 # If we're not streaming, read the content so we can retry any timeout ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/client.py in _request(self, method, str_or_url, params, data, json, cookies, headers, skip_auto_headers, auth, allow_redirects, max_redirects, compress, chunked, expect100, raise_for_status, read_until_eof, proxy, proxy_auth, timeout, verify_ssl, fingerprint, ssl_context, ssl, proxy_headers, trace_request_ctx) 474 req, 475 traces=traces, --> 476 timeout=real_timeout 477 ) 478 except asyncio.TimeoutError as exc: ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in connect(self, req, traces, timeout) 520 521 try: --> 522 proto = await self._create_connection(req, traces, timeout) 523 if self._closed: 524 proto.close() ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _create_connection(self, req, traces, timeout) 852 else: 853 _, proto = await self._create_direct_connection( --> 854 req, traces, timeout) 855 856 return proto ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _create_direct_connection(self, req, traces, timeout, client_error) 990 else: 991 assert last_exc is not None --> 992 raise last_exc 993 994 async def _create_proxy_connection( ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _create_direct_connection(self, req, traces, timeout, client_error) 972 server_hostname=hinfo['hostname'] if sslcontext else None, 973 local_addr=self._local_addr, --> 974 req=req, client_error=client_error) 975 except ClientConnectorError as exc: 976 last_exc = exc ~/**/**/*********/venv/lib/python3.7/site-packages/aiohttp/connector.py in _wrap_create_connection(self, req, timeout, client_error, *args, **kwargs) 925 except cert_errors as exc: 926 raise ClientConnectorCertificateError( --> 927 req.connection_key, exc) from exc 928 except ssl_errors as exc: 929 raise ClientConnectorSSLError(req.connection_key, exc) from exc ClientConnectorCertificateError: Cannot connect to host portal.sso.ap-southeast-1.amazonaws.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')] ```

However, when I use the normal boto3 for the above two functions - upload_fileobj & get_presigned_url. It works fine.

terricain commented 3 years ago

ClientConnectorCertificateError: Cannot connect to host portal.sso.ap-southeast-1.amazonaws.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')]

This looks related to how SSO works, this is most likely an issue with aiobotocore. Can you try with that library and see if you get the same issue?