Wrong type for hdrs in urllib.error.HTTPError

[bdrung]

Example code code.py:

import urllib.error
raise urllib.error.HTTPError("https://example.com/", 502, "Bad Gateway", hdrs={}, fp=None)

mypy complains:

code.py:2: error: Argument "hdrs" to "HTTPError" has incompatible type "Dict[<nothing>, <nothing>]"; expected "Message"  [arg-type]

Looking at the Python stdlib code, hdrs is always a dict[str, str].

[srittau]

Where do you see that in the stdlib code? The only instances I see where HTTPError is instantiated are in urllib/request.py, which just passes the headers on from the various handlers. Those handlers get passed HTTPMessage objects - at least according to our stubs. Those are not dicts.

[bdrung]

All those functions in urllib.requests raises urllib.error.HTTPError with hdrs=headers:

• HTTPRedirectHandler.redirect_request
• HTTPRedirectHandler.http_error_302
• AbstractDigestAuthHandler.http_error_auth_reqed
• URLopener.http_error_default
• FancyURLopener.redirect_internal

urllib.requests.Request sets:

self.headers = {}

and then adds headers with:

    def add_header(self, key, val):
        # useful for something like authentication
        self.headers[key.capitalize()] = val

[srittau]

I'm not sure why our stubs say the handler methods expect a Message. The documentation says: "... hdrs will be a mapping object ..." and I couldn't find any relevant urllib code that uses (HTTP)Message. (urlretrieve() and noheaders() return a Message, but they are not called from within urllib.)

So I think the best option is to indeed change HTTPError.hdrs and the arguments to the methods to MutableMapping or Mapping. Any PRs to improve the situation are welcome!

[aminalaee]

Checking the history, the PR which added this is: https://github.com/python/typeshed/pull/5854

And trying a sample code, looks like (HTTP)Message is the correct type, which also implements some Mapping methods, but it's not the other way around.

import urllib.error
import urllib.request

try:
    urllib.request.urlopen('https://jigsaw.w3.org/HTTP/Basic/')
except urllib.error.HTTPError as ex:
    print(type(ex.headers))  # <class 'http.client.HTTPMessage'>

Which is according to the docs: https://docs.python.org/3/library/urllib.error.html#urllib.error.HTTPError.headers https://docs.python.org/3/library/http.client.html#httpmessage-objects

@srittau Do you still think the change is required?