search

requests / toolbelt

A toolbelt of useful classes and functions to be used with python-requests
https://toolbelt.readthedocs.org
Other
998 stars 186 forks source link

InvalidSchema raised unnecessary when url_based is activated #305

Closed briner closed 3 years ago

briner commented 3 years ago

I have the impression that the base_url is not working as expected

base_url="https://an.domain-name.com/images/"
sub_path="sha256:102ab2db1ad671545c0ace25463c4e3c45f9b15e319d3a00a1b2b085293c27fb"
full_url=f"{base_url}{sub_path}"

from requests_toolbelt import sessions
session = sessions.BaseUrlSession(base_url=base_url)

# this will work
session.get(full_url)
  # <Response [200]>

# but this will fail, as it interpret sh256 as an invalid schema
session.get(sub_path)
  # Traceback (most recent call last):
  #   File "<stdin>", line 1, in <module>
  #   File "/home/brinerc/code/ptl-anchore/venv/lib64/python3.6/site-packages/requests/sessions.py", line 555, in get
  #     return self.request('GET', url, **kwargs)
  #   File "/home/brinerc/code/ptl-anchore/venv/lib64/python3.6/site-packages/requests_toolbelt/sessions.py", line 65, in request
  #     method, url, *args, **kwargs
  #   File "/home/brinerc/code/ptl-anchore/venv/lib64/python3.6/site-packages/requests/sessions.py", line 542, in request
  #     resp = self.send(prep, **send_kwargs)
  #   File "/home/brinerc/code/ptl-anchore/venv/lib64/python3.6/site-packages/requests/sessions.py", line 649, in send
  #     adapter = self.get_adapter(url=request.url)
  #   File "/home/brinerc/code/ptl-anchore/venv/lib64/python3.6/site-packages/requests/sessions.py", line 742, in get_adapter
  #     raise InvalidSchema("No connection adapters were found for {!r}".format(url))
  # requests.exceptions.InvalidSchema: No connection adapters were found for 'sha256:102ab2db1ad671545c0ace25463c4e3c45f9b15e319d3a00a1b2b085293c27fb'

So giving the subpath sha256:102ab2db1ad671545c0ace25463c4e3c45f9b15e319d3a00a1b2b085293c27fb for the session.get make it fail !

the versions:

Python 3.6.8
requests           2.25.1
requests-toolbelt  0.9.1
sigmavirus24 commented 3 years ago

This is not a bug in the BaseUrlSession, it's a bug in the fact that you've made assumptions that are invalid because of how URL Relative Resolution is defined in RFC 3986 and is implemented in the standard library. Observe:

Python 3.9.1 (default, Jan 20 2021, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urljoin
>>> urljoin("https://an.domain-name.com/images/", "sha256:102ab2db1ad671545c0ace25463c4e3c45f9b15e319d3a00a1b2b085293c27fb")
'sha256:102ab2db1ad671545c0ace25463c4e3c45f9b15e319d3a00a1b2b085293c27fb'

And we rely on that https://github.com/requests/toolbelt/blob/26fe2b3e23f6d766e3db22dffe1c0a1e992c1f70/requests_toolbelt/sessions.py#L68-L70

You could use "./sha256:102ab2db1ad671545c0ace25463c4e3c45f9b15e319d3a00a1b2b085293c27fb" instead to get the behaviour you want

briner commented 3 years ago

happy, to have found a solution.

Many thank for your help and your time.

Regards