mindflayer / python-mocket

a socket mock framework - for all kinds of socket animals, web-clients included
BSD 3-Clause "New" or "Revised" License
279 stars 41 forks source link

Does not work with urllib3 >= 2.0.0 #199

Closed oschwald closed 1 year ago

oschwald commented 1 year ago

As of urlib3 2.0.0, we are seeing failures such as the following:

======================================================================
ERROR: webservice_test (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: webservice_test
Traceback (most recent call last):
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/home/greg/MaxMind/GeoIP2-python/tests/webservice_test.py", line 16, in <module>
    from mocket import Mocket  # type: ignore
  File "/home/greg/MaxMind/GeoIP2-python/.eggs/mocket-3.11.0-py3.10.egg/mocket/__init__.py", line 1, in <module>
    from .async_mocket import async_mocketize
  File "/home/greg/MaxMind/GeoIP2-python/.eggs/mocket-3.11.0-py3.10.egg/mocket/async_mocket.py", line 1, in <module>
    from .mocket import Mocketizer
  File "/home/greg/MaxMind/GeoIP2-python/.eggs/mocket-3.11.0-py3.10.egg/mocket/mocket.py", line 18, in <module>
    from urllib3.util.ssl_ import wrap_socket as urllib3_wrap_socket
ImportError: cannot import name 'wrap_socket' from 'urllib3.util.ssl_' (/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/util/ssl_.py)

This is for the tests in the geoip2 package, but I don't know that the failure is particular to anything we are doing.

s-t-e-v-e-n-k commented 1 year ago

urllib3 never actually exported wrap_socket, it was imported from ssl, that is what mocket is pulling in. That import was removed for 2.0.

mindflayer commented 1 year ago

I'll have a look at it. Thanks for reporting the issue.

mindflayer commented 1 year ago

@oschwald could you please test out the code from the above PR? Don't worry if tests are failing, they rely A LOT on httpbin.org and it's returning some BAD GATEWAY errors here and there. I have a plan to replace it with a Docker container, but it was not an easy change and I had no time to complete it.

oschwald commented 1 year ago

Thanks for making the PR! Unfortunately, I am getting errors like this one with it when using urllib3 2.0.2:

======================================================================
ERROR: test_weird_body_error (tests.webservice_test.TestClient)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/mocket/mocket.py", line 701, in wrapper
    return test(*args, **kwargs)
  File "/home/greg/MaxMind/GeoIP2-python/tests/webservice_test.py", line 177, in test_weird_body_error
    self.run_client(self.client.country("1.2.3.8"))
  File "/home/greg/MaxMind/GeoIP2-python/geoip2/webservice.py", line 462, in country
    Country, self._response_for("country", geoip2.models.Country, ip_address)
  File "/home/greg/MaxMind/GeoIP2-python/geoip2/webservice.py", line 489, in _response_for
    response = self._session.get(uri, proxies=self._proxies, timeout=self._timeout)
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1092, in _validate_conn
    conn.connect()
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connection.py", line 635, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connection.py", line 728, in _ssl_wrap_socket_and_match_hostname
    context = create_urllib3_context(
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 282, in create_urllib3_context
    context.minimum_version = TLSVersion.TLSv1_2
  File "/home/greg/.pyenv/versions/3.10.0/lib/python3.10/ssl.py", line 603, in minimum_version
    super(SSLContext, SSLContext).minimum_version.__set__(self, value)
AttributeError: 'super' object has no attribute 'minimum_version'

urllib3 1.26.4 works fine with the changes in the PR.

mindflayer commented 1 year ago

Perfect, I was asking to try it out just because I was not able to run the whole pipeline. I'll try to tackle all the other problems that appear and release a new version ASAP.

mindflayer commented 1 year ago

Enjoy the new version of Mocket: https://pypi.org/project/mocket/3.11.1/

oschwald commented 1 year ago

Thanks! The new version seems to work well.