Release 3.0.3 breaking Python < 3.10 users

[tobias-urdin]

The 3.0.3 release breaks Python < 3.10 users due to the str | None typing syntax that is only available in 3.10 and later, but python-requires says supported Python versions is >=3.8. This should use typing.Union[str, None] instead.

The relevant commit: https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01#diff-32b889e713f9b5346cff198fd82aabe162ae8c529f008fa2a7a8fc6dbff17570R11

TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Environment:

• Python version: 3.9
• Werkzeug version: 3.0.3

[davidism]

I can't reproduce this issue, all tests pass on all supported Python versions: https://github.com/pallets/werkzeug/actions/runs/10269498182/job/28415084983 When reporting an issue, be sure to include a minimal reproducible example and full traceback.

[tobias-urdin]

Example job that fails with Python 3.9: https://github.com/gnocchixyz/gnocchi/actions/runs/10370049894/job/28707101343?pr=1395

Trace from above job:

Traceback (most recent call last):
  File "/usr/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/github/workspace/gnocchi/tests/test_rest.py", line 36, in <module>
    from gnocchi.rest import api
  File "/github/workspace/gnocchi/rest/api.py", line 32, in <module>
    import werkzeug.http
  File "/usr/lib/python3/dist-packages/werkzeug/__init__.py", line 2, in <module>
    from .test import Client as Client
  File "/usr/lib/python3/dist-packages/werkzeug/test.py", line 42, in <module>
    from .utils import get_content_type
  File "/usr/lib/python3/dist-packages/werkzeug/utils.py", line 25, in <module>
    from .wsgi import wrap_file
  File "/usr/lib/python3/dist-packages/werkzeug/wsgi.py", line 11, in <module>
    from .sansio import utils as _sansio_utils
  File "/usr/lib/python3/dist-packages/werkzeug/sansio/utils.py", line 9, in <module>
    def host_is_trusted(hostname: str | None, trusted_list: t.Iterable[str]) -> bool:
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Reproduce:

$ python3 --version
Python 3.8.10

import typing as t

def host_is_trusted(hostname: str | None, trusted_list: t.Iterable[str]) -> bool:
    print('hello world')

host_is_trusted(None, [])

$ python3 test.py 
Traceback (most recent call last):
  File "test.py", line 3, in <module>
    def host_is_trusted(hostname: str | None, trusted_list: t.Iterable[str]) -> bool:
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Suggested fix:

def host_is_trusted(hostname: t.Union[str, None], trusted_list: t.Iterable[str]) -> bool:

Can you please reopen?

[Callum027]

The likely reason why the error is not occurring is because postponed annotation evaluation is enabled (from __future__ import annotations).

Whenever you do an operation that causes the annotations to be evaluated, you get the errors:

$ python -c 'from typing import get_type_hints; from werkzeug.sansio.utils import host_is_trusted; print(get_type_hints(host_is_trusted))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/typing.py", line 1264, in get_type_hints
    value = _eval_type(value, globalns, localns)
  File "/usr/lib/python3.8/typing.py", line 270, in _eval_type
    return t._evaluate(globalns, localns)
  File "/usr/lib/python3.8/typing.py", line 518, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

And it's not just host_is_trusted. Every function that uses PEP-604 will need this change.

I'm still not sure why annotations in Werkzeug are being evaluated in the Gnocchi tests specifically, the test suite Gnocchi uses might be doing something that causes Python to evaluate them on import. Either way, I believe this is a valid bug.

[RonnyPfannschmidt]

There's a back port package that enables evaluating modern type annotations on python before they were supported, I recommend trying if installing fixes the issue as a workaround

[tobias-urdin]

There's a back port package that enables evaluating modern type annotations on python before they were supported, I recommend trying if installing fixes the issue as a workaround

Can you please elaborate what this "back port package" is and how I can test it?

I don't think closing this is fair to users if saying that this version supports older Python versions, "install this as well" for a older version does not give confidence that upstream actually support the older version.

[ThiefMaster]

Pretty sure closing was because initially we could not reproduce it. Give people some time. ;) @RonnyPfannschmidt clearly said that this is a possible workaround, not a full solution...

[RonnyPfannschmidt]

https://pypi.org/project/eval-type-backport/

[RonnyPfannschmidt]

does werkzeug promise runtime type annotations an any way?

if yes the code in question should be ported to t.Optional

alternatively a recommendation should be made to use the backport if necessary

personally i'd go towards recommending the backport and running pyupgrade

[davidism]

No, we do not promise runtime type evalutation. We are using features available in standard Python, Mypy, and Pyright. from __future__ import annotations defers evaluation of annotations. MyPy and Pyright allow using typing features from the latest Python version when using this mode.