4.2.1: `pyupgrade --py39-plus` generated patch causes test suite failures

kloczek commented 3 months ago

Things to check first

[X] I have searched the existing issues and didn't find my bug already reported there
[X] I have checked that my bug is still present in the latest release

Typeguard version

4.2.1

Python version

3.10.14

What happened?

Looks like current code is not ready to be upgraded to python 3.9+ using pyupgrade --py39-plus and it causes test suite fails.

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 rootdir: /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1 configfile: pyproject.toml testpaths: tests plugins: typeguard-4.2.1, asyncio-0.23.7 asyncio: mode=strict collected 455 items tests/mypy/test_type_annotations.py .. [ 0%] tests/test_checkers.py ..................................................................s.s........................................................F.........F...................... [ 35%] ............FF.................. [ 42%] tests/test_importhook.py ... [ 42%] tests/test_instrumentation.py ........xxx..............................................xxx...................................... [ 64%] tests/test_plugins.py . [ 64%] tests/test_pytest_plugin.py .. [ 65%] tests/test_suppression.py ....... [ 66%] tests/test_transformer.py ................................................s............................... [ 84%] tests/test_typechecked.py ..................................................... [ 95%] tests/test_union_transformer.py ..F..F.....F [ 98%] tests/test_utils.py ..... [ 99%] tests/test_warn_on_error.py .. [100%] ========================================================================================= FAILURES ========================================================================================== ______________________________________________________________________________ TestUnion.test_typing_type_fail ______________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_checkers.py:780: in test_typing_type_fail pytest.raises(TypeCheckError, check_type, 1, Union[str, Collection]).match( E AssertionError: Regex pattern did not match. E Regex: 'int did not match any element in the union:\n str: is not an instance of str\n Collection: is not an instance of collections.abc.Collection' E Input: 'int did not match any element in the union:\n str: is not an instance of str\n collections.abc.Collection: is not an instance of collections.abc.Collection' _______________________________________________________________________ TestTypevar.test_collection_constraints_fail ________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_checkers.py:857: in test_collection_constraints_fail pytest.raises(TypeCheckError, check_type, {1, 2}, TTypingConstrained).match( E AssertionError: Regex pattern did not match. E Regex: 'set does not match any of the constraints \\(List\\[int\\], AbstractSet\\[str\\]\\)' E Input: 'set does not match any of the constraints (list[int], AbstractSet[str])' _______________________________________________________________________________ TestRecursiveType.test_valid ________________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_checkers.py:1054: in test_valid check_type({"a": [1, 2, 3]}, JSONType) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_functions.py:106: in check_type check_type_internal(value, expected_type, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:777: in check_type_internal checker(value, origin_type, args, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:416: in check_union check_type_internal(value, type_, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:777: in check_type_internal checker(value, origin_type, args, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:235: in check_mapping check_type_internal(v, value_type, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:784: in check_type_internal warnings.warn( E typeguard.TypeHintWarning: Skipping type check against 'JSONType'; this looks like a string-form forward reference imported from another module ________________________________________________________________________________ TestRecursiveType.test_fail ________________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_checkers.py:1076: in test_fail check_type({"a": (1, 2, 3)}, JSONType) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_functions.py:106: in check_type check_type_internal(value, expected_type, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:777: in check_type_internal checker(value, origin_type, args, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:416: in check_union check_type_internal(value, type_, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:777: in check_type_internal checker(value, origin_type, args, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:235: in check_mapping check_type_internal(v, value_type, memo) /home/tkloczko/rpmbuild/BUILDROOT/python-typeguard-4.2.1-2.fc37.x86_64/usr/lib/python3.10/site-packages/typeguard/_checkers.py:784: in check_type_internal warnings.warn( E typeguard.TypeHintWarning: Skipping type check against 'JSONType'; this looks like a string-form forward reference imported from another module _____________________________________________________ test_union_transformer[str | Union[int | bytes, set]-Union[str, int, bytes, Set]] _____________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_union_transformer.py:45: in test_union_transformer assert evaluated_repr == expected E AssertionError: assert 'Union[str, int, bytes, set]' == 'Union[str, int, bytes, Set]' E E - Union[str, int, bytes, Set] E ? ^ E + Union[str, int, bytes, set] E ? ^ ______________________________________ test_union_transformer[str | int | Callable[[], bytes | set]-Union[str, int, Callable[[], Union[bytes, Set]]]] _______________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_union_transformer.py:45: in test_union_transformer assert evaluated_repr == expected E AssertionError: assert 'Union[str, i...bytes, set]]]' == 'Union[str, i...bytes, Set]]]' E E - Union[str, int, Callable[[], Union[bytes, Set]]] E ? ^ E + Union[str, int, Callable[[], Union[bytes, set]]] E ? ^ __________________________________________________________________ test_union_transformer[tuple[int, ...]-Tuple[int, ...]] __________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_union_transformer.py:45: in test_union_transformer assert evaluated_repr == expected E AssertionError: assert 'tuple[int, ...]' == 'Tuple[int, ...]' E E - Tuple[int, ...] E ? ^ E + tuple[int, ...] E ? ^ ========================================================================================= XFAILURES ========================================================================================= ___________________________________________________________________________ test_inner_class_method[typechecked] ____________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_instrumentation.py:102: in test_inner_class_method retval = dummymodule.Outer().create_inner() /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/dummymodule.py:116: in create_inner def create_inner(self) -> "Inner": E NameError: name 'Inner' is not defined _________________________________________________________________________ test_inner_class_classmethod[typechecked] _________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_instrumentation.py:108: in test_inner_class_classmethod retval = dummymodule.Outer.create_inner_classmethod() /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/dummymodule.py:120: in create_inner_classmethod def create_inner_classmethod(cls) -> "Inner": E NameError: name 'Inner' is not defined ________________________________________________________________________ test_inner_class_staticmethod[typechecked] _________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_instrumentation.py:114: in test_inner_class_staticmethod retval = dummymodule.Outer.create_inner_staticmethod() /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/dummymodule.py:124: in create_inner_staticmethod def create_inner_staticmethod() -> "Inner": E NameError: name 'Inner' is not defined ____________________________________________________________________________ test_inner_class_method[importhook] ____________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_instrumentation.py:102: in test_inner_class_method retval = dummymodule.Outer().create_inner() /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/dummymodule.py:116: in create_inner def create_inner(self) -> "Inner": E NameError: name 'Inner' is not defined _________________________________________________________________________ test_inner_class_classmethod[importhook] __________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_instrumentation.py:108: in test_inner_class_classmethod retval = dummymodule.Outer.create_inner_classmethod() /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/dummymodule.py:120: in create_inner_classmethod def create_inner_classmethod(cls) -> "Inner": E NameError: name 'Inner' is not defined _________________________________________________________________________ test_inner_class_staticmethod[importhook] _________________________________________________________________________ /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/test_instrumentation.py:114: in test_inner_class_staticmethod retval = dummymodule.Outer.create_inner_staticmethod() /home/tkloczko/rpmbuild/BUILD/typeguard-4.2.1/tests/dummymodule.py:124: in create_inner_staticmethod def create_inner_staticmethod() -> "Inner": E NameError: name 'Inner' is not defined ================================================================================== short test summary info ================================================================================== SKIPPED [1] tests/test_checkers.py:504: 'NotRequired' not found in 'typing' SKIPPED [1] tests/test_checkers.py:516: 'NotRequired' not found in 'typing' SKIPPED [1] tests/test_transformer.py:1219: Requires Python < 3.10 XFAIL tests/test_instrumentation.py::test_inner_class_method[typechecked] - No workaround for this has been implemented yet XFAIL tests/test_instrumentation.py::test_inner_class_classmethod[typechecked] - No workaround for this has been implemented yet XFAIL tests/test_instrumentation.py::test_inner_class_staticmethod[typechecked] - No workaround for this has been implemented yet XFAIL tests/test_instrumentation.py::test_inner_class_method[importhook] - No workaround for this has been implemented yet XFAIL tests/test_instrumentation.py::test_inner_class_classmethod[importhook] - No workaround for this has been implemented yet XFAIL tests/test_instrumentation.py::test_inner_class_staticmethod[importhook] - No workaround for this has been implemented yet FAILED tests/test_checkers.py::TestUnion::test_typing_type_fail - AssertionError: Regex pattern did not match. FAILED tests/test_checkers.py::TestTypevar::test_collection_constraints_fail - AssertionError: Regex pattern did not match. FAILED tests/test_checkers.py::TestRecursiveType::test_valid - typeguard.TypeHintWarning: Skipping type check against 'JSONType'; this looks like a string-form forward reference imported from another module FAILED tests/test_checkers.py::TestRecursiveType::test_fail - typeguard.TypeHintWarning: Skipping type check against 'JSONType'; this looks like a string-form forward reference imported from another module FAILED tests/test_union_transformer.py::test_union_transformer[str | Union[int | bytes, set]-Union[str, int, bytes, Set]] - AssertionError: assert 'Union[str, int, bytes, set]' == 'Union[str, int, bytes, Set]' FAILED tests/test_union_transformer.py::test_union_transformer[str | int | Callable[[], bytes | set]-Union[str, int, Callable[[], Union[bytes, Set]]]] - AssertionError: assert 'Union[str, i...bytes, set]]]' == 'Union[str, i...bytes, Set]]]' FAILED tests/test_union_transformer.py::test_union_transformer[tuple[int, ...]-Tuple[int, ...]] - AssertionError: assert 'tuple[int, ...]' == 'Tuple[int, ...]' =================================================================== 7 failed, 439 passed, 3 skipped, 6 xfailed in 11.54s ==================================================================== ```

This issue is not urgent but it would be good to make typeguard code ready for pypugrade (python 3.8 will be EOSed next month) or report to pypugrade that it messes with code in some cases.

How can we reproduce the bug?

filter all code by find . -name \*.py | xargs pyupgrade --py39-plus and than build and test module

Here is patch generated by pyuprade

```patch --- a/src/typeguard/_importhook.py +++ b/src/typeguard/_importhook.py @@ -10,7 +10,8 @@ from inspect import isclass from os import PathLike from types import CodeType, ModuleType, TracebackType -from typing import Sequence, TypeVar +from typing import TypeVar +from collections.abc import Sequence from unittest.mock import patch from ._config import global_config --- a/src/typeguard/_transformer.py +++ b/src/typeguard/_transformer.py @@ -421,12 +421,12 @@ # Only treat the first argument to typing.Annotated as a potential # forward reference items = cast( - typing.List[expr], + list[expr], [self.visit(slice_value.elts[0])] + slice_value.elts[1:], ) else: items = cast( - typing.List[expr], + list[expr], [self.visit(item) for item in slice_value.elts], ) @@ -748,10 +748,7 @@ if node.args.vararg: annotation_ = self._convert_annotation(node.args.vararg.annotation) if annotation_: - if sys.version_info >= (3, 9): - container = Name("tuple", ctx=Load()) - else: - container = self._get_import("typing", "Tuple") + container = Name("tuple", ctx=Load()) subscript_slice: Tuple | Index = Tuple( [ @@ -770,10 +767,7 @@ if node.args.kwarg: annotation_ = self._convert_annotation(node.args.kwarg.annotation) if annotation_: - if sys.version_info >= (3, 9): - container = Name("dict", ctx=Load()) - else: - container = self._get_import("typing", "Dict") + container = Name("dict", ctx=Load()) subscript_slice = Tuple( [ --- a/src/typeguard/_union_transformer.py +++ b/src/typeguard/_union_transformer.py @@ -21,11 +21,11 @@ from typing import Any, Dict, FrozenSet, List, Set, Tuple, Union type_substitutions = { - "dict": Dict, - "list": List, - "tuple": Tuple, - "set": Set, - "frozenset": FrozenSet, + "dict": dict, + "list": list, + "tuple": tuple, + "set": set, + "frozenset": frozenset, "Union": Union, } --- a/src/typeguard/_checkers.py +++ b/src/typeguard/_checkers.py @@ -19,11 +19,8 @@ Dict, ForwardRef, List, - Mapping, - MutableMapping, NewType, Optional, - Sequence, Set, TextIO, Tuple, @@ -31,6 +28,7 @@ TypeVar, Union, ) +from collections.abc import Mapping, MutableMapping, Sequence from unittest.mock import Mock try: @@ -61,12 +59,12 @@ SubclassableAny = Any else: from typing_extensions import ( - Annotated, NotRequired, TypeAlias, get_args, get_origin, ) + from typing import Annotated from typing_extensions import Any as SubclassableAny if sys.version_info >= (3, 10): @@ -77,16 +75,15 @@ from typing_extensions import ParamSpec TypeCheckerCallable: TypeAlias = Callable[ - [Any, Any, Tuple[Any, ...], TypeCheckMemo], Any + [Any, Any, tuple[Any, ...], TypeCheckMemo], Any ] TypeCheckLookupCallback: TypeAlias = Callable[ - [Any, Tuple[Any, ...], Tuple[Any, ...]], Optional[TypeCheckerCallable] + [Any, tuple[Any, ...], tuple[Any, ...]], Optional[TypeCheckerCallable] ] checker_lookup_functions: list[TypeCheckLookupCallback] = [] -generic_alias_types: tuple[type, ...] = (type(List), type(List[Any])) -if sys.version_info >= (3, 9): - generic_alias_types += (types.GenericAlias,) +generic_alias_types: tuple[type, ...] = (type(list), type(list[Any])) +generic_alias_types += (types.GenericAlias,) # Sentinel @@ -212,7 +209,7 @@ args: tuple[Any, ...], memo: TypeCheckMemo, ) -> None: - if origin_type is Dict or origin_type is dict: + if origin_type is dict or origin_type is dict: if not isinstance(value, dict): raise TypeCheckError("is not a dict") if origin_type is MutableMapping or origin_type is collections.abc.MutableMapping: @@ -523,12 +520,12 @@ ) -> None: if origin_type.__bound__ is not None: annotation = ( - Type[origin_type.__bound__] if subclass_check else origin_type.__bound__ + type[origin_type.__bound__] if subclass_check else origin_type.__bound__ ) check_type_internal(value, annotation, memo) elif origin_type.__constraints__: for constraint in origin_type.__constraints__: - annotation = Type[constraint] if subclass_check else constraint + annotation = type[constraint] if subclass_check else constraint try: check_type_internal(value, annotation, memo) except TypeCheckError: @@ -768,7 +765,7 @@ # Compatibility hack to distinguish between unparametrized and empty tuple # (tuple[()]), necessary due to https://github.com/python/cpython/issues/91137 - if origin_type in (tuple, Tuple) and annotation is not Tuple and not args: + if origin_type in (tuple, tuple) and annotation is not tuple and not args: args = ((),) else: origin_type = annotation @@ -801,12 +798,12 @@ collections.abc.Callable: check_callable, complex: check_number, dict: check_mapping, - Dict: check_mapping, + dict: check_mapping, float: check_number, frozenset: check_set, IO: check_io, list: check_list, - List: check_list, + list: check_list, typing.Literal: check_literal, Mapping: check_mapping, MutableMapping: check_mapping, @@ -817,12 +814,12 @@ collections.abc.Sequence: check_sequence, collections.abc.Set: check_set, set: check_set, - Set: check_set, + set: check_set, TextIO: check_io, tuple: check_tuple, - Tuple: check_tuple, + tuple: check_tuple, + type: check_class, type: check_class, - Type: check_class, Union: check_union, } if sys.version_info >= (3, 10): @@ -853,7 +850,7 @@ elif is_typeddict(origin_type): return check_typed_dict elif isclass(origin_type) and issubclass( - origin_type, Tuple # type: ignore[arg-type] + origin_type, tuple # type: ignore[arg-type] ): # NamedTuple return check_tuple --- a/tests/__init__.py +++ b/tests/__init__.py @@ -1,6 +1,5 @@ from typing import ( AbstractSet, - Collection, Dict, Generic, List, @@ -11,12 +10,13 @@ Union, runtime_checkable, ) +from collections.abc import Collection T_Foo = TypeVar("T_Foo") TBound = TypeVar("TBound", bound="Parent") TConstrained = TypeVar("TConstrained", "Parent", int) -TTypingConstrained = TypeVar("TTypingConstrained", List[int], AbstractSet[str]) +TTypingConstrained = TypeVar("TTypingConstrained", list[int], AbstractSet[str]) TIntStr = TypeVar("TIntStr", int, str) TIntCollection = TypeVar("TIntCollection", int, Collection[int]) TParent = TypeVar("TParent", bound="Parent") @@ -28,9 +28,9 @@ id: int -JSONType = Union[str, float, bool, None, List["JSONType"], Dict[str, "JSONType"]] +JSONType = Union[str, float, bool, None, list["JSONType"], dict[str, "JSONType"]] myint = NewType("myint", int) -mylist = NewType("mylist", List[int]) +mylist = NewType("mylist", list[int]) class FooGeneric(Generic[T_Foo]): --- a/tests/dummymodule.py +++ b/tests/dummymodule.py @@ -5,13 +5,10 @@ from typing import ( TYPE_CHECKING, Any, - AsyncGenerator, Callable, Dict, - Generator, List, Literal, - Sequence, Tuple, Type, TypeVar, @@ -20,6 +17,7 @@ no_type_check_decorator, overload, ) +from collections.abc import AsyncGenerator, Generator, Sequence from typeguard import ( CollectionCheckStrategy, @@ -218,7 +216,7 @@ @typechecked -def multi_assign_single_value() -> Tuple[int, float, complex]: +def multi_assign_single_value() -> tuple[int, float, complex]: x: int y: float z: complex @@ -227,7 +225,7 @@ @typechecked -def multi_assign_iterable() -> Tuple[Sequence[int], Sequence[float], Sequence[complex]]: +def multi_assign_iterable() -> tuple[Sequence[int], Sequence[float], Sequence[complex]]: x: Sequence[int] y: Sequence[float] z: Sequence[complex] @@ -236,14 +234,14 @@ @typechecked -def unpacking_assign() -> Tuple[int, str]: +def unpacking_assign() -> tuple[int, str]: x: int x, y = (1, "foo") return x, y @typechecked -def unpacking_assign_generator() -> Tuple[int, str]: +def unpacking_assign_generator() -> tuple[int, str]: def genfunc(): yield 1 yield "foo" @@ -254,7 +252,7 @@ @typechecked -def unpacking_assign_star_with_annotation() -> Tuple[int, List[bytes], str]: +def unpacking_assign_star_with_annotation() -> tuple[int, list[bytes], str]: x: int z: str x, *y, z = (1, b"abc", b"bah", "foo") @@ -262,9 +260,9 @@ @typechecked -def unpacking_assign_star_no_annotation(value: Any) -> Tuple[int, List[bytes], str]: +def unpacking_assign_star_no_annotation(value: Any) -> tuple[int, list[bytes], str]: x: int - y: List[bytes] + y: list[bytes] z: str x, *y, z = value return x, y, z @@ -281,7 +279,7 @@ @typechecked(collection_check_strategy=CollectionCheckStrategy.ALL_ITEMS) -def override_collection_check_strategy(value: List[int]) -> None: +def override_collection_check_strategy(value: list[int]) -> None: pass @@ -299,7 +297,7 @@ @typechecked def typed_variable_args( *args: str, **kwargs: int -) -> Tuple[Tuple[str, ...], Dict[str, int]]: +) -> tuple[tuple[str, ...], dict[str, int]]: return args, kwargs @@ -317,9 +315,9 @@ @typechecked def guarded_type_hint_subscript_nested( - x: List["Imaginary[int]"], -) -> List["Imaginary[int]"]: - y: List[Imaginary[int]] = x + x: list["Imaginary[int]"], +) -> list["Imaginary[int]"]: + y: list[Imaginary[int]] = x return y @@ -336,10 +334,10 @@ @typechecked -def typevar_forwardref(x: Type[T]) -> T: +def typevar_forwardref(x: type[T]) -> T: return x() -def never_called(x: List["NonExistentType"]) -> List["NonExistentType"]: # noqa: F821 +def never_called(x: list["NonExistentType"]) -> list["NonExistentType"]: # noqa: F821 """Regression test for #335.""" return x --- a/tests/mypy/test_type_annotations.py +++ b/tests/mypy/test_type_annotations.py @@ -18,7 +18,7 @@ ] -def get_mypy_cmd(filename: str) -> List[str]: +def get_mypy_cmd(filename: str) -> list[str]: return ["mypy", "--strict", filename] @@ -34,7 +34,7 @@ return output -def get_expected_errors() -> Dict[int, str]: +def get_expected_errors() -> dict[int, str]: """ Extract the expected errors from comments in the negative examples file. """ @@ -53,7 +53,7 @@ return expected -def get_mypy_errors() -> Dict[int, str]: +def get_mypy_errors() -> dict[int, str]: """ Extract the errors from running mypy on the negative examples file. """ --- a/tests/test_transformer.py +++ b/tests/test_transformer.py @@ -6,10 +6,7 @@ from typeguard._transformer import TypeguardTransformer -if sys.version_info >= (3, 9): - from ast import unparse -else: - pytest.skip("Requires Python 3.9 or newer", allow_module_level=True) +from ast import unparse def test_arguments_only() -> None: @@ -1166,12 +1163,8 @@ ) TypeguardTransformer().visit(node) - if sys.version_info < (3, 9): - extra_import = "from typing import Tuple\n" - tuple_type = "Tuple" - else: - extra_import = "" - tuple_type = "tuple" + extra_import = "" + tuple_type = "tuple" assert ( unparse(node) @@ -1202,12 +1195,8 @@ ) TypeguardTransformer().visit(node) - if sys.version_info < (3, 9): - extra_import = "from typing import Dict\n" - dict_type = "Dict" - else: - extra_import = "" - dict_type = "dict" + extra_import = "" + dict_type = "dict" assert ( unparse(node) --- a/tests/test_typechecked.py +++ b/tests/test_typechecked.py @@ -6,15 +6,10 @@ from textwrap import dedent from typing import ( Any, - AsyncGenerator, - AsyncIterable, - AsyncIterator, Dict, - Generator, - Iterable, - Iterator, List, ) +from collections.abc import AsyncGenerator, AsyncIterable, AsyncIterator, Generator, Iterable, Iterator from unittest.mock import Mock import pytest @@ -83,7 +78,7 @@ def test_generator_annotated(self): @typechecked - def genfunc() -> Generator[int, str, List[str]]: + def genfunc() -> Generator[int, str, list[str]]: val1 = yield 2 val2 = yield 3 val3 = yield 4 @@ -599,7 +594,7 @@ ], ) def test_typechecked_disabled_in_optimized_mode( - tmp_path: Path, flags: List[str], expected_return_code: int + tmp_path: Path, flags: list[str], expected_return_code: int ): code = dedent( """ @@ -629,7 +624,7 @@ # Regression test for #362 @typechecked class A: - def foo(self) -> Dict[str, Any]: + def foo(self) -> dict[str, Any]: return {} A().foo() --- a/tests/test_warn_on_error.py +++ b/tests/test_warn_on_error.py @@ -16,7 +16,7 @@ def test_typechecked(monkeypatch, recwarn): @typechecked - def foo() -> List[int]: + def foo() -> list[int]: return ["aa"] # type: ignore[list-item] monkeypatch.setattr(config, "typecheck_fail_callback", warn_on_error) --- a/tests/test_checkers.py +++ b/tests/test_checkers.py @@ -11,18 +11,13 @@ AnyStr, BinaryIO, Callable, - Collection, ContextManager, Dict, ForwardRef, FrozenSet, - Iterator, List, Literal, - Mapping, - MutableMapping, Optional, - Sequence, Set, TextIO, Tuple, @@ -30,6 +25,7 @@ TypeVar, Union, ) +from collections.abc import Collection, Iterator, Mapping, MutableMapping, Sequence import pytest @@ -74,10 +70,7 @@ else: from typing_extensions import Concatenate, ParamSpec, TypeGuard -if sys.version_info >= (3, 9): - from typing import Annotated -else: - from typing_extensions import Annotated +from typing import Annotated P = ParamSpec("P") @@ -405,17 +398,17 @@ class TestDict: def test_bad_type(self): - pytest.raises(TypeCheckError, check_type, 5, Dict[str, int]).match( + pytest.raises(TypeCheckError, check_type, 5, dict[str, int]).match( "int is not a dict" ) def test_bad_key_type(self): - pytest.raises(TypeCheckError, check_type, {1: 2}, Dict[str, int]).match( + pytest.raises(TypeCheckError, check_type, {1: 2}, dict[str, int]).match( "key 1 of dict is not an instance of str" ) def test_bad_value_type(self): - pytest.raises(TypeCheckError, check_type, {"x": "a"}, Dict[str, int]).match( + pytest.raises(TypeCheckError, check_type, {"x": "a"}, dict[str, int]).match( "value of key 'x' of dict is not an instance of int" ) @@ -424,7 +417,7 @@ TypeCheckError, check_type, {"x": 1, 3: 2}, - Dict[str, int], + dict[str, int], collection_check_strategy=CollectionCheckStrategy.ALL_ITEMS, ).match("key 3 of dict is not an instance of str") @@ -433,7 +426,7 @@ TypeCheckError, check_type, {"x": 1, "y": "a"}, - Dict[str, int], + dict[str, int], collection_check_strategy=CollectionCheckStrategy.ALL_ITEMS, ).match("value of key 'y' of dict is not an instance of int") @@ -443,7 +436,7 @@ for key in self: yield key, self[key] - check_type(CustomDict(a=1), Dict[str, int]) + check_type(CustomDict(a=1), dict[str, int]) class TestTypedDict: @@ -542,18 +535,18 @@ class TestList: def test_bad_type(self): - pytest.raises(TypeCheckError, check_type, 5, List[int]).match( + pytest.raises(TypeCheckError, check_type, 5, list[int]).match( "int is not a list" ) def test_first_check_success(self): - check_type(["aa", "bb", 1], List[str]) + check_type(["aa", "bb", 1], list[str]) def test_first_check_empty(self): - check_type([], List[str]) + check_type([], list[str]) def test_first_check_fail(self): - pytest.raises(TypeCheckError, check_type, ["bb"], List[int]).match( + pytest.raises(TypeCheckError, check_type, ["bb"], list[int]).match( "list is not an instance of int" ) @@ -562,7 +555,7 @@ TypeCheckError, check_type, [1, 2, "bb"], - List[int], + list[int], collection_check_strategy=CollectionCheckStrategy.ALL_ITEMS, ).match("list is not an instance of int") @@ -635,16 +628,16 @@ class TestSet: def test_bad_type(self): - pytest.raises(TypeCheckError, check_type, 5, Set[int]).match("int is not a set") + pytest.raises(TypeCheckError, check_type, 5, set[int]).match("int is not a set") def test_valid(self): - check_type({1, 2}, Set[int]) + check_type({1, 2}, set[int]) def test_first_check_empty(self): - check_type(set(), Set[int]) + check_type(set(), set[int]) def test_first_check_fail(self, sample_set: set): - pytest.raises(TypeCheckError, check_type, sample_set, Set[int]).match( + pytest.raises(TypeCheckError, check_type, sample_set, set[int]).match( "set is not an instance of int" ) @@ -653,26 +646,26 @@ TypeCheckError, check_type, {1, 2, "bb"}, - Set[int], + set[int], collection_check_strategy=CollectionCheckStrategy.ALL_ITEMS, ).match("set is not an instance of int") class TestFrozenSet: def test_bad_type(self): - pytest.raises(TypeCheckError, check_type, 5, FrozenSet[int]).match( + pytest.raises(TypeCheckError, check_type, 5, frozenset[int]).match( "int is not a frozenset" ) def test_valid(self): - check_type(frozenset({1, 2}), FrozenSet[int]) + check_type(frozenset({1, 2}), frozenset[int]) def test_first_check_empty(self): - check_type(frozenset(), FrozenSet[int]) + check_type(frozenset(), frozenset[int]) def test_first_check_fail(self, sample_set: set): pytest.raises( - TypeCheckError, check_type, frozenset(sample_set), FrozenSet[int] + TypeCheckError, check_type, frozenset(sample_set), frozenset[int] ).match("set is not an instance of int") def test_full_check_fail(self): @@ -680,12 +673,12 @@ TypeCheckError, check_type, frozenset({1, 2, "bb"}), - FrozenSet[int], + frozenset[int], collection_check_strategy=CollectionCheckStrategy.ALL_ITEMS, ).match("set is not an instance of int") def test_set_against_frozenset(self, sample_set: set): - pytest.raises(TypeCheckError, check_type, sample_set, FrozenSet[int]).match( + pytest.raises(TypeCheckError, check_type, sample_set, frozenset[int]).match( "set is not a frozenset" ) @@ -693,7 +686,7 @@ @pytest.mark.parametrize( "annotated_type", [ - pytest.param(Tuple, id="typing"), + pytest.param(tuple, id="typing"), pytest.param( tuple, id="builtin", @@ -891,11 +884,11 @@ class TestType: - @pytest.mark.parametrize("annotation", [pytest.param(Type), pytest.param(type)]) + @pytest.mark.parametrize("annotation", [pytest.param(type), pytest.param(type)]) def test_unparametrized(self, annotation: Any): check_type(TestNewType, annotation) - @pytest.mark.parametrize("annotation", [pytest.param(Type), pytest.param(type)]) + @pytest.mark.parametrize("annotation", [pytest.param(type), pytest.param(type)]) def test_unparametrized_fail(self, annotation: Any): pytest.raises(TypeCheckError, check_type, 1, annotation).match( "int is not a class" @@ -905,10 +898,10 @@ "value", [pytest.param(Parent, id="exact"), pytest.param(Child, id="subclass")] ) def test_parametrized(self, value): - check_type(value, Type[Parent]) + check_type(value, type[Parent]) def test_parametrized_fail(self): - pytest.raises(TypeCheckError, check_type, int, Type[str]).match( + pytest.raises(TypeCheckError, check_type, int, type[str]).match( "class int is not a subclass of str" ) @@ -916,17 +909,17 @@ "value", [pytest.param(str, id="str"), pytest.param(int, id="int")] ) def test_union(self, value): - check_type(value, Type[Union[str, int, list]]) + check_type(value, type[Union[str, int, list]]) def test_union_any(self): - check_type(list, Type[Union[str, int, Any]]) + check_type(list, type[Union[str, int, Any]]) def test_any(self): - check_type(list, Type[Any]) + check_type(list, type[Any]) def test_union_fail(self): pytest.raises( - TypeCheckError, check_type, dict, Type[Union[str, int, list]] + TypeCheckError, check_type, dict, type[Union[str, int, list]] ).match( "class dict did not match any element in the union:\n" " str: is not a subclass of str\n" @@ -936,15 +929,14 @@ def test_union_typevar(self): T = TypeVar("T", bound=Parent) - check_type(Child, Type[T]) + check_type(Child, type[T]) - @pytest.mark.parametrize("check_against", [type, Type[Any]]) + @pytest.mark.parametrize("check_against", [type, type[Any]]) def test_generic_aliase(self, check_against): - if sys.version_info >= (3, 9): - check_type(dict[str, str], check_against) + check_type(dict[str, str], check_against) - check_type(Dict, check_against) - check_type(Dict[str, str], check_against) + check_type(dict, check_against) + check_type(dict[str, str], check_against) class TestIO: @@ -1004,7 +996,7 @@ pass check_type(Foo(), RuntimeProtocol) - check_type(Foo, Type[RuntimeProtocol]) + check_type(Foo, type[RuntimeProtocol]) def test_protocol_warns_on_static(self): class Foo: @@ -1023,7 +1015,7 @@ with pytest.warns( UserWarning, match=r"Typeguard cannot check the StaticProtocol protocol.*" ) as warning: - check_type(Foo, Type[StaticProtocol]) + check_type(Foo, type[StaticProtocol]) assert warning.list[0].filename == __file__ @@ -1038,7 +1030,7 @@ pytest.raises(TypeCheckError, check_type, Foo(), RuntimeProtocol).match( f"{clsname} is not compatible with the RuntimeProtocol protocol" ) - pytest.raises(TypeCheckError, check_type, Foo, Type[RuntimeProtocol]).match( + pytest.raises(TypeCheckError, check_type, Foo, type[RuntimeProtocol]).match( f"class {clsname} is not compatible with the RuntimeProtocol protocol" ) @@ -1052,7 +1044,7 @@ f"the RuntimeProtocol protocol" ) pytest.raises(TypeCheckError, check_type, Foo(), RuntimeProtocol).match(pattern) - pytest.raises(TypeCheckError, check_type, Foo, Type[RuntimeProtocol]).match( + pytest.raises(TypeCheckError, check_type, Foo, type[RuntimeProtocol]).match( pattern ) @@ -1159,7 +1151,7 @@ def test_return_checked_value(): value = {"foo": 1} - assert check_type(value, Dict[str, int]) is value + assert check_type(value, dict[str, int]) is value def test_imported_str_forward_ref(): ```

agronholm commented 3 months ago

Pyupgrade is being overly zealous here, and I would use ruff instead of pyupgrade for this anyway.

agronholm commented 3 months ago

Also it'll be around 5 months until 3.8 reaches end of life: https://devguide.python.org/versions/#supported-versions

kloczek commented 3 months ago

Pyupgrade is being overly zealous here, and I would use ruff instead of pyupgrade for this anyway.

ruff is not performing any python version code updates. It does many other thing so both tools should be used (combined). If you will try to use ruff on current code it will generate empty patch.

kloczek commented 3 months ago

Also it'll be around 5 months until 3.8 reaches end of life: https://devguide.python.org/versions/#supported-versions

Yep my mistake .. nevertheless it is relatively close to 3.8 EOS.

agronholm commented 3 months ago

Pyupgrade is being overly zealous here, and I would use ruff instead of pyupgrade for this anyway.

ruff is not performing any python version code updates. It does many other thing so both tools should be used (combined). If you will try to use ruff on current code it will generate empty patch.

Did you update requires-python in pyproject.toml before running ruff?

kloczek commented 3 months ago

Did you update requires-python in pyproject.toml before running ruff?

Just tested that

[tkloczko@pers-jacek SOURCES]$ ls -l python-typeguard-ruff.patch python-typeguard-pyupgrade_py39.patch
-rw-r--r-- 1 tkloczko tkloczko 24613 May 25 14:56 python-typeguard-pyupgrade_py39.patch
-rw-r--r-- 1 tkloczko tkloczko  3602 May 25 14:56 python-typeguard-ruff.patch

agronholm commented 3 months ago

Typeguard is somewhat of a special case in that it needs to still use the "deprecated" types from typing, so an upgrade should not make too many changes in that regard.

agronholm commented 3 months ago

Why is this an issue at all anyway?

kloczek commented 3 months ago

And after apply pyupgrade_py39 generated ruff sill generates even bigger patch

-rw-r--r-- 1 tkloczko tkloczko  4985 May 25 14:58 python-typeguard-ruff.patch

Why is this an issue at all anyway?

Quote from opening this ticket

Looks like current code is not ready to be upgraded to python 3.9+ using pyupgrade --py39-plus and it causes test suite fails.

agronholm commented 3 months ago

Why would you perform such an action anyways before 3.8 is EOL? I routinely upgrade my projects in such a manner when I drop support for an obsolete Python version.

agronholm commented 3 months ago

In fact, this happens automatically when I bump python_requires.

kloczek commented 3 months ago

I think that above shows clearly that ruff and pyupgrade are doing different checks.

Why would you perform such an action anyways before 3.8 is EOL? I routinely upgrade my projects in such a manner when I drop support for an obsolete Python version.

Because in distro on which I'm working currently is used python 3.11 (soon will be migration to 3.12) and many code can be dropped with assumption that generated packages will be used with exact version.

Nevertheless pyupgrade generates patch which breaks test suite.

In fact, this happens automatically when I bump python_requires.

python_requires is used with setup.{py|cfg}. With pyproject.toml is used requires-python.

agronholm commented 3 months ago

If you want to modify typeguard downstream, you're free to do so but you will do so at your own risk and expense. Pyupgrade making a patch that fails the test suite isn't my problem. When the time comes, I will use ruff to upgrade the code base as needed.

kloczek commented 3 months ago

If you want to modify typeguard downstream,

I have no enough knowledge about your module. This is why I've reported what I found.

You can ignore this issue and fact you didn't know that ruff is not doing what pyupgrade is doing but issue still remains ..

agronholm / typeguard