search

testcontainers / testcontainers-python

Testcontainers is a Python library that providing a friendly API to run Docker container. It is designed to create runtime environment to use during your automatic tests.
https://testcontainers-python.readthedocs.io/en/latest/
Apache License 2.0
1.51k stars 281 forks source link

Bug: Incompatibility with Requests 2.32.1 causing connection to Docker daemon to fail #577

Closed NikiforovG closed 3 months ago

NikiforovG commented 3 months ago

Describe the bug

I experienced an issue when running pytest tests involving the Testcontainers Python library in my environment. The Docker client library was unable to connect to the Docker daemon and threw an error: 'Error while fetching server API version: Not supported URL scheme http+docker'.

To Reproduce

The issue seems to present itself when the following conditions are met:

Rolling back the requests library to version 2.31.0 resolved my issue, which leads me to believe that the issue is with requests 2.32.1 compatibility.

Runtime environment

Python 3.11.7 pytest==8.2.0 testcontainers==3.4.4 requests==2.32.1 (issue occurs) requests==2.31.0 (issue does not occur)

``` test setup failed self = request = , stream = False, timeout = 60, verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest ` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) ` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: > conn = self._get_connection(request, verify, proxies=proxies, cert=cert) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/requests/adapters.py:532: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/requests/adapters.py:400: in _get_connection conn = self.poolmanager.connection_from_host( ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/urllib3/poolmanager.py:304: in connection_from_host return self.connection_from_context(request_context) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = request_context = {'block': False, 'cert_reqs': 'CERT_REQUIRED', 'host': 'localhost', 'maxsize': 10, ...} def connection_from_context( self, request_context: dict[str, typing.Any] ) -> HTTPConnectionPool: """ Get a :class:`urllib3.connectionpool.ConnectionPool` based on the request context. ``request_context`` must at least contain the ``scheme`` key and its value must be a key in ``key_fn_by_scheme`` instance variable. """ if "strict" in request_context: warnings.warn( "The 'strict' parameter is no longer needed on Python 3+. " "This will raise an error in urllib3 v2.1.0.", DeprecationWarning, ) request_context.pop("strict") scheme = request_context["scheme"].lower() pool_key_constructor = self.key_fn_by_scheme.get(scheme) if not pool_key_constructor: > raise URLSchemeUnknown(scheme) E urllib3.exceptions.URLSchemeUnknown: Not supported URL scheme http+docker ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/urllib3/poolmanager.py:326: URLSchemeUnknown During handling of the above exception, another exception occurred: self = def _retrieve_server_version(self): try: > return self.version(api_version=False)["ApiVersion"] ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/docker/api/client.py:213: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/docker/api/daemon.py:181: in version return self._result(self._get(url), json=True) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/docker/utils/decorators.py:44: in inner return f(self, *args, **kwargs) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/docker/api/client.py:236: in _get return self.get(url, **self._set_request_timeout(kwargs)) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/requests/sessions.py:602: in get return self.request("GET", url, **kwargs) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = request = , stream = False, timeout = 60, verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest ` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) ` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self._get_connection(request, verify, proxies=proxies, cert=cert) except LocationValueError as e: > raise InvalidURL(e, request=request) E requests.exceptions.InvalidURL: Not supported URL scheme http+docker ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/requests/adapters.py:534: InvalidURL The above exception was the direct cause of the following exception: @pytest.fixture(scope="session") def postgres_container() -> Generator[PostgresContainer, None, None]: > with PostgresContainer("postgres:16", dbname='test', driver=None) as postgres: tests/conftest.py:10: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/testcontainers/postgres/__init__.py:61: in __init__ super().__init__(image=image, **kwargs) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/testcontainers/core/container.py:46: in __init__ self._docker = DockerClient(**(docker_client_kw or {})) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/testcontainers/core/docker_client.py:56: in __init__ self.client = docker.from_env(**kwargs) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/docker/client.py:94: in from_env return cls( ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/docker/client.py:45: in __init__ self.api = APIClient(*args, **kwargs) ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/docker/api/client.py:197: in __init__ self._version = self._retrieve_server_version() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def _retrieve_server_version(self): try: return self.version(api_version=False)["ApiVersion"] except KeyError as ke: raise DockerException( 'Invalid response from docker daemon: key "ApiVersion"' ' is missing.' ) from ke except Exception as e: > raise DockerException( f'Error while fetching server API version: {e}' ) from e E docker.errors.DockerException: Error while fetching server API version: Not supported URL scheme http+docker ../../../../Library/Caches/pypoetry/virtualenvs/task-consumer-u30wUliX-py3.11/lib/python3.11/site-packages/docker/api/client.py:220: DockerException ```
ErwinJapie commented 3 months ago

Running into the exact same issue as of today.

alexanderankin commented 3 months ago

Any relevant issues on requests library side?

On Tue, May 21, 2024, 2:30 PM Erwin Boshart @.***> wrote:

Running into the exact issue as of today.

— Reply to this email directly, view it on GitHub https://github.com/testcontainers/testcontainers-python/issues/577#issuecomment-2123203687, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACECGJCP7I7NYCIIYVKPU2TZDOHCRAVCNFSM6AAAAABICC6H6CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRTGIYDGNRYG4 . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>

alexanderankin commented 3 months ago

related issues:

alexanderankin commented 3 months ago

there has been a release https://pypi.org/project/docker/#history - perhaps if someone can retest and update here if its still an issue, then we can close

ErwinJapie commented 3 months ago

I can confirm this issue has been resolved for us after the update.

kiview commented 3 months ago

Thanks you @ErwinJapie, I'll close this now, since from my understanding, it is essentially resolved through the docker-py release.