search

Mbed-TLS / mbedtls-framework

TF-PSA-Crypto and Mbed TLS version-independent build and test framework
Other
2 stars 16 forks source link

Incorrect type annotations and mypy ≥0.960 failure in macro_collector.py #50

Open gilles-peskine-arm opened 2 months ago

gilles-peskine-arm commented 2 months ago

scripts/mbedtls_framework/macro_collector.py fails to typecheck with mypy ≥0.960.

framework/scripts/mbedtls_framework/macro_collector.py:488: error: No overload variant of "sub" matches argument types "Pattern[Any]", "bytes", "str"
framework/scripts/mbedtls_framework/macro_collector.py:488: note: Possible overload variants:
framework/scripts/mbedtls_framework/macro_collector.py:488: note:     def sub(pattern: Union[str, Pattern[str]], repl: Union[str, Callable[[Match[str]], str]], string: str, count: int = ..., flags: Union[int, RegexFlag] = ...) -> str
framework/scripts/mbedtls_framework/macro_collector.py:488: note:     def sub(pattern: Union[bytes, Pattern[bytes]], repl: Union[Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]], Callable[[Match[bytes]], Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]]]], string: Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]], count: int = ..., flags: Union[int, RegexFlag] = ...) -> bytes

The problem is that mypy's knowledge of read_file_lines is not precise enough, or perhaps outright wrong. The type annotation on read_file_lines.__iter__ says that it's an Iterator[str], but it's actually an iterator over bytes when the context manager was initialized with binary=True. Earlier versions of mypy were fine with that, but since mypy 0.960, they're complaining, I think rightfully, I think thanks to improvements in how overloads for open are handled.

The code isn't wrong, but it has a dependent type, which is too hard for mypy.

The goal of this issue is to fix macro_collector.py so that our code typechecks under modern mypy.