search

minio / minio-py

MinIO Client SDK for Python
https://docs.min.io/docs/python-client-quickstart-guide.html
Apache License 2.0
851 stars 325 forks source link

urllib3.connectionpool:Retrying when try list_objects() #1418

Closed vedricopane closed 6 months ago

vedricopane commented 6 months ago

I am trying to get data from bucket using python and error connectionpool:Retrying.

My code :

client = Minio(endpoint='url-endpoint',
               access_key='access_key',
               secret_key='secret_key', secure=True)

objects = client.list_objects(
    "miniotest", prefix="data/", recursive=True,
)
for obj in objects:
    print(obj)

This error got when running "looping" section.

Error : WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7caf9289e260>: Failed to resolve 'apiminio.telkom.co.id' ([Errno -2] Name or service not known)")': /miniotest?location= WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7caf9289ffa0>: Failed to resolve 'apiminio.telkom.co.id' ([Errno -2] Name or service not known)")': /miniotest?location= WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7caf9289fd30>: Failed to resolve 'apiminio.telkom.co.id' ([Errno -2] Name or service not known)")': /miniotest?location= WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7caf9289fa90>: Failed to resolve 'apiminio.telkom.co.id' ([Errno -2] Name or service not known)")': /miniotest?location= WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7caf9289fa00>: Failed to resolve 'apiminio.telkom.co.id' ([Errno -2] Name or service not known)")': /miniotest?location=

gaierror Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/urllib3/connection.py in _new_conn(self) 202 try: --> 203 sock = connection.create_connection( 204 (self._dns_host, self.port),

20 frames gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

NameResolutionError Traceback (most recent call last) NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7caf9289cdc0>: Failed to resolve 'apiminio.telkom.co.id' ([Errno -2] Name or service not known)

The above exception was the direct cause of the following exception:

MaxRetryError Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace) 513 if new_retry.is_exhausted(): 514 reason = error or ResponseError(cause) --> 515 raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] 516 517 log.debug("Incremented Retry for (url='%s'): %r", url, new_retry)

MaxRetryError: HTTPSConnectionPool(host='apiminio.telkom.co.id', port=443): Max retries exceeded with url: /miniotest?location= (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7caf9289cdc0>: Failed to resolve 'apiminio.telkom.co.id' ([Errno -2] Name or service not known)"))

balamurugana commented 6 months ago

By default, the default http client in minio-py retries on certain errors. If you don't want to do that, you could pass your custom http client to Minio class.