search

encode / httpx

A next generation HTTP client for Python. 🦋
https://www.python-httpx.org/
BSD 3-Clause "New" or "Revised" License
13.28k stars 844 forks source link

0.12.0 PyPI wheel contains both public- and private-name modules #864

Closed kbakk closed 4 years ago

kbakk commented 4 years ago

The following works in httpx 0.11.1:

In [1]: import httpx 
   ...: from httpx.exceptions import InvalidURL                                                                                                                             

In [2]: try: 
   ...:     httpx.get("foo.bar") 
   ...: except InvalidURL: 
   ...:     pass 
   ...:

In 0.12.0 the exception isn't caught:

In [1]: import httpx 
   ...: from httpx.exceptions import InvalidURL                                                                                                                             

In [2]: try: 
   ...:     httpx.get("foo.bar") 
   ...: except InvalidURL: 
   ...:     pass 
   ...:                                                                                                                                                                     
---------------------------------------------------------------------------
InvalidURL                                Traceback (most recent call last)
<ipython-input-2-87135a63c42c> in <module>
      1 try:
----> 2     httpx.get("foo.bar")
      3 except InvalidURL:
      4     pass
      5 

~/.venv/lib/python3.7/site-packages/httpx/_api.py in get(url, params, headers, cookies, auth, allow_redirects, cert, verify, timeout, trust_env)
    166         verify=verify,
    167         timeout=timeout,
--> 168         trust_env=trust_env,
    169     )
    170 

~/.venv/lib/python3.7/site-packages/httpx/_api.py in request(method, url, params, data, files, json, headers, cookies, auth, timeout, allow_redirects, verify, cert, trust_env)
     92             cookies=cookies,
     93             auth=auth,
---> 94             allow_redirects=allow_redirects,
     95         )
     96 

~/.venv/lib/python3.7/site-packages/httpx/_client.py in request(self, method, url, data, files, json, params, headers, cookies, auth, allow_redirects, timeout)
    566             params=params,
    567             headers=headers,
--> 568             cookies=cookies,
    569         )
    570         return self.send(

~/.venv/lib/python3.7/site-packages/httpx/_client.py in build_request(self, method, url, data, files, json, params, headers, cookies)
    196         Build and return a request instance.
    197         """
--> 198         url = self.merge_url(url)
    199         headers = self.merge_headers(headers)
    200         cookies = self.merge_cookies(cookies)

~/.venv/lib/python3.7/site-packages/httpx/_client.py in merge_url(self, url)
    216         to create the URL used for the outgoing request.
    217         """
--> 218         url = self.base_url.join(relative_url=url)
    219         if url.scheme == "http" and hstspreload.in_hsts_preload(url.host):
    220             port = None if url.port == 80 else url.port

~/.venv/lib/python3.7/site-packages/httpx/_models.py in join(self, relative_url)
    227         """
    228         if self.is_relative_url:
--> 229             return URL(relative_url)
    230 
    231         # We drop any fragment portion, because RFC 3986 strictly

~/.venv/lib/python3.7/site-packages/httpx/_models.py in __init__(self, url, allow_relative, params)
    104         if not allow_relative:
    105             if not self.scheme:
--> 106                 raise InvalidURL("No scheme included in URL.")
    107             if not self.host:
    108                 raise InvalidURL("No host included in URL.")

InvalidURL: No scheme included in URL.

This works though:

In [3]: import httpx 
   ...: from httpx._exceptions import InvalidURL                                                                                                                            

In [4]: try: 
   ...:     httpx.get("foo.bar") 
   ...: except InvalidURL: 
   ...:     pass 
   ...:
florimondmanca commented 4 years ago

Hi @kbakk,

There is no httpx.exceptions module anymore in 0.12 (it's be moved to a private httpx._exceptions module, as you noted).

So your situation might be due to a spurious cache issue… I'd suggest you try uninstalling HTTPX completely (pip uninstall httpx -y), then reinstalling a fresh copy of 0.12.

In general you should only be using the top-level httpx module. We moved to private module names to enforce this more clearly (see #772). Using httpx.InvalidURL instead of importing the exception class, I'm not able to reproduce this behavior, even if installing 0.11.1 and then upgrading to 0.12.

florimondmanca commented 4 years ago

I'll close for now, but if you keep encountering issues if switching to httpx.<...> then feel free to report. :-) Thanks!

StephenBrown2 commented 4 years ago

To extract a working code sample from florimondmanca's prose, this should work:

In [1]: import httpx

In [2]: try:
   ...:     httpx.get("foo.bar")
   ...: except httpx.InvalidURL:
   ...:     pass
   ...:

As should this, though less recommended:

In [1]: import httpx
   ...: from httpx import InvalidURL

In [2]: try:
   ...:     httpx.get("foo.bar")
   ...: except InvalidURL:
   ...:     pass
   ...:
kbakk commented 4 years ago

Thanks for the quick response.

I tried to remove and add it back as suggested. I'm still able to reproduce it.

I downloaded the wheel and tar (from https://pypi.org/project/httpx/#files), and found that the wheel contains the exceptions.py file, while the tar.gz don't:

unzip -l httpx-0.12.0-py3-none-any.whl | grep exceptions
     2981  03-06-2020 09:33   httpx/_exceptions.py
     3061  01-08-2020 09:21   httpx/exceptions.py

tar -ztvf httpx-0.12.0.tar.gz | grep exceptions
-rw-r--r--  0 tomchristie staff    2981 Mar  6 10:33 httpx-0.12.0/httpx/_exceptions.py
florimondmanca commented 4 years ago

Thanks; reopened, as I was able to reproduce by downloading the wheel for 0.12.0. It actually contain everything in duplicate:

$ unzip -l httpx-0.12.0-py3-none-any.whl
Archive:  httpx-0.12.0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
     1852  03-06-2020 09:33   httpx/__init__.py
      108  03-09-2020 10:18   httpx/__version__.py
    10370  03-05-2020 10:36   httpx/_api.py
     8525  03-05-2020 10:36   httpx/_auth.py
    46105  03-06-2020 09:33   httpx/_client.py
    12117  03-06-2020 09:33   httpx/_config.py
    10907  03-06-2020 09:33   httpx/_content_streams.py
     8882  02-26-2020 09:40   httpx/_decoders.py
     2981  03-06-2020 09:33   httpx/_exceptions.py
    39223  03-06-2020 09:33   httpx/_models.py
     5181  02-26-2020 09:40   httpx/_status_codes.py
    11582  03-05-2020 16:05   httpx/_utils.py
    10460  01-09-2020 09:24   httpx/api.py
     8487  01-08-2020 09:21   httpx/auth.py
    46297  01-09-2020 09:24   httpx/client.py
    11406  01-09-2020 09:24   httpx/config.py
    10913  01-06-2020 09:35   httpx/content_streams.py
     8633  01-08-2020 09:21   httpx/decoders.py
     3061  01-08-2020 09:21   httpx/exceptions.py
    39087  01-09-2020 09:24   httpx/models.py
        0  08-16-2019 10:36   httpx/py.typed
     5181  12-02-2019 10:58   httpx/status_codes.py
    11579  01-08-2020 09:21   httpx/utils.py
        0  02-26-2020 09:40   httpx/_backends/__init__.py
     9408  02-26-2020 09:40   httpx/_backends/asyncio.py
     1548  02-26-2020 09:40   httpx/_backends/auto.py
     3414  02-26-2020 09:40   httpx/_backends/base.py
     5800  02-26-2020 09:40   httpx/_backends/trio.py
        0  02-26-2020 09:40   httpx/_dispatch/__init__.py
     4010  03-06-2020 09:33   httpx/_dispatch/asgi.py
     1728  03-06-2020 09:33   httpx/_dispatch/base.py
     5600  03-06-2020 09:33   httpx/_dispatch/connection.py
     7413  03-06-2020 09:33   httpx/_dispatch/connection_pool.py
     7169  03-06-2020 09:33   httpx/_dispatch/http11.py
    11134  03-06-2020 09:33   httpx/_dispatch/http2.py
     7428  03-06-2020 09:33   httpx/_dispatch/proxy_http.py
     4040  03-06-2020 09:33   httpx/_dispatch/urllib3.py
     3467  03-06-2020 09:33   httpx/_dispatch/wsgi.py
        0  12-31-2019 12:19   httpx/backends/__init__.py
     9405  01-08-2020 09:21   httpx/backends/asyncio.py
     1547  01-08-2020 09:21   httpx/backends/auto.py
     3413  01-08-2020 09:21   httpx/backends/base.py
     5797  01-08-2020 09:21   httpx/backends/trio.py
        0  01-09-2020 09:24   httpx/dispatch/__init__.py
     4007  01-09-2020 09:24   httpx/dispatch/asgi.py
     1726  01-09-2020 09:24   httpx/dispatch/base.py
     5596  01-08-2020 09:21   httpx/dispatch/connection.py
     7408  01-08-2020 09:21   httpx/dispatch/connection_pool.py
     7163  12-31-2019 15:06   httpx/dispatch/http11.py
    11128  12-31-2019 15:06   httpx/dispatch/http2.py
     7423  01-08-2020 09:21   httpx/dispatch/proxy_http.py
     4031  01-09-2020 09:24   httpx/dispatch/urllib3.py
     3464  01-09-2020 09:24   httpx/dispatch/wsgi.py
     1518  03-09-2020 10:19   httpx-0.12.0.dist-info/LICENSE.md
    20600  03-09-2020 10:19   httpx-0.12.0.dist-info/METADATA
       92  03-09-2020 10:19   httpx-0.12.0.dist-info/WHEEL
       38  03-09-2020 10:19   httpx-0.12.0.dist-info/top_level.txt
     4550  03-09-2020 10:19   httpx-0.12.0.dist-info/RECORD
---------                     -------
   484002                     58 files

I tried rebuilding the wheel from the 0.12.0 tag, and the result only contains _*.py modules/packages, as expected:

$ pip install wheel
$ python setup.py build bdist_wheel
$ unzip -l dist/httpx-0.12.0-py3-none-any.whl| grep exceptions
     2981  03-14-2020 09:34   httpx/_exceptions.py

So uh… I don't know why, but the 0.12.0 wheel uploaded to PyPI has all items in duplicate. 🤔

It's not possible to reupload versions (see https://github.com/pypa/packaging-problems/issues/74), so I don't think there's much we can do apart from issuing 0.12.1 soon with a clean build. At least we're aware of this now…

@tomchristie Any thoughts?

florimondmanca commented 4 years ago

Updated the issue description to reflect these latest findings.

tomchristie commented 4 years ago

Okay doke, let roll a re-release due to packaging issues.

florimondmanca commented 4 years ago

@tomchristie Would that be a 0.12.1 release?

tomchristie commented 4 years ago

@florimondmanca I guess so, yes. A guess a helpful PR related to this would be adding whatever git command to the scripts/clean file we need in order to delete anything that's not in source control. (That probably would have prevented this one.)