tracker: enforce typings and enable mypy checks in github actions

shubhbapna commented 2 months ago

We have already been seeing type mismatch of between what is being passed to a function vs what the function parameter expects especially in the case of version where we have been passing strings as well as Version objects.

Enforcing types will help us sync everything and should prevent similar future issues.

[x] fix typings for src/fromager/clickext.py (no errors detected)
[x] fix typings for src/fromager/constraints.py #227
[x] fix typings for src/fromager/context.py #229
[x] fix typings for src/fromager/dependencies.py #234
[x] fix typings for src/fromager/example_override.py (no errors detected)
[x] fix typings for src/fromager/external_commands.py #234
[x] fix typings for src/fromager/extras_provider.py #273
[x] fix typings for src/fromager/finders.py (callable function related errors only see tasks below)
[x] fix typings for src/fromager/hooks.py #245
[x] fix typings for src/fromager/__main__.py #246
[x] fix typings for src/fromager/overrides.py #237
[x] fix typings for src/fromager/requirements_file.py #231
[x] fix typings for src/fromager/resolver.py #300
[ ] fix typings for src/fromager/sdist.py
[x] fix typings for src/fromager/server.py #247
[ ] fix typings for src/fromager/settings.py #257
[ ] fix typings for src/fromager/sources.py
[x] fix typings for src/fromager/tarballs.py #242
[x] fix typings for src/fromager/vendor_rust.py #239
[x] fix typings for src/fromager/wheels.py #240
[x] fix typings for src/fromager/commands/bootstrap.py #264
[x] fix typings for src/fromager/commands/build_order.py #265
[x] fix typings for src/fromager/commands/build.py #249
[x] fix typings for src/fromager/commands/canonicalize.py #238
[x] fix typings for src/fromager/commands/download_sequence.py #238
[x] fix typings for src/fromager/commands/__init__.py (no errors detected)
[x] fix typings for src/fromager/commands/list_overrides.py #238
[x] src/fromager/commands/server.py #238
[x] src/fromager/commands/step.py #248
[x] src/fromager/progress.py #297

After all of the above are done:

[ ] Remove typings.Any. Define type aliases wherever needed. Files affected: constraints.py, context.py, external_commands.py, dependencies.py, overrides.py, wheels.py, progress.py, build_order.py
[ ] Add a callable protocol for override functions in overrides.py. Use generics to provide type hints for the return value of invoke. Check if type hints are working wherever invoke is being called.
[ ] enable mypy checks (strict mode) in github actions

[ ] Until the regression in pyproject_hooks 1.1.0 is fixed, ask mypy to ignore the following errors in dependencies.py

src/fromager/dependencies.py:119: error: Returning Any from function declared to return "Iterable[str]"  [no-any-return]
src/fromager/dependencies.py:165: error: Returning Any from function declared to return "Iterable[str]"  [no-any-return]

[ ] Change extra_environ to dict[str, str]
[ ] Change *args and **kwargs annotation to only type the item / value
[ ] Change version strings to Version object
[ ] https://github.com/python-wheel-build/fromager/pull/249#discussion_r1691811775 https://github.com/python-wheel-build/fromager/pull/249#discussion_r1692454269

shubhbapna commented 2 months ago

Currently mypy has detected 70 errors:

src/fromager/overrides.py:46: error: Function "fn" could always be true in boolean context  [truthy-function]
src/fromager/overrides.py:145: error: Incompatible return value type (got "None", expected "Callable[..., Any]")  [return-value]
src/fromager/overrides.py:148: error: Incompatible return value type (got "None", expected "Callable[..., Any]")  [return-value]
src/fromager/settings.py:46: error: Argument 1 to "_parse" has incompatible type "TextIOWrapper"; expected "str"  [arg-type]
src/fromager/requirements_file.py:38: error: List comprehension has incompatible type List[dict[str, object]]; expected List[Environment]  [misc]
src/fromager/requirements_file.py:41: error: Argument 1 to "evaluate" of "Marker" has incompatible type "Environment"; expected "dict[str, str] | None"  [arg-type]
src/fromager/finders.py:33: error: Function "sdist_name_func" could always be true in boolean context  [truthy-function]
src/fromager/finders.py:123: error: Function "sdir_name_func" could always be true in boolean context  [truthy-function]
src/fromager/constraints.py:64: error: Argument 1 to "_organize_constraints" has incompatible type "list[Any]"; expected "dict[Any, Any]"  [arg-type]
src/fromager/context.py:54: error: Need type annotation for "_build_stack" (hint: "_build_stack: list[<type>] = ...")  [var-annotated]
src/fromager/context.py:55: error: Need type annotation for "_build_requirements" (hint: "_build_requirements: set[<type>] = ...")  [var-annotated]
src/fromager/context.py:62: error: Need type annotation for "_seen_requirements" (hint: "_seen_requirements: set[<type>] = ...")  [var-annotated]
src/fromager/context.py:69: error: List item 1 has incompatible type "str | None"; expected "str"  [list-item]
src/fromager/context.py:81: error: Incompatible return value type (got "bool", expected "tuple[Any, ...]")  [return-value]
src/fromager/commands/build_order.py:71: error: Incompatible types in assignment (expression has type "TextIO | Any", variable has type "TextIOWrapper")  [assignment]
src/fromager/commands/build_order.py:102: error: Need type annotation for "dist_to_input_file"  [var-annotated]
src/fromager/commands/build_order.py:113: error: Incompatible types in assignment (expression has type "TextIO | Any", variable has type "TextIOWrapper")  [assignment]
src/fromager/commands/build_order.py:178: error: Need type annotation for "nodes" (hint: "nodes: dict[<type>, <type>] = ...")  [var-annotated]
src/fromager/commands/build_order.py:213: error: Incompatible types in assignment (expression has type "TextIO | Any", variable has type "TextIOWrapper")  [assignment]
src/fromager/server.py:33: error: Argument "directory" to "LoggingHTTPRequestHandler" has incompatible type "Path"; expected "str | None"  [arg-type]
src/fromager/resolver.py:89: error: Incompatible types in assignment (expression has type "list[Requirement]", variable has type "None")  [assignment]
src/fromager/resolver.py:90: error: Incompatible return value type (got "None", expected "list[Requirement]")  [return-value]
src/fromager/resolver.py:137: error: Need type annotation for "tags" (hint: "tags: set[<type>] = ...")  [var-annotated]
src/fromager/resolver.py:140: error: Incompatible types in assignment (expression has type "frozenset[Tag]", variable has type "set[Any]")  [assignment]
src/fromager/resolver.py:167: error: Argument "extras" to "Candidate" has incompatible type "tuple[str]"; expected "dict[Any, Any] | None"  [arg-type]
src/fromager/resolver.py:180: error: Incompatible return value type (got "Message[str, str]", expected "EmailMessage")  [return-value]
src/fromager/resolver.py:208: error: Argument 1 to "sorted" has incompatible type "set[str] | dict[Any, Any] | None"; expected "Iterable[Any]"  [arg-type]
src/fromager/resolver.py:277: error: Argument 1 to "sorted" has incompatible type "set[str] | dict[Any, Any] | None"; expected "Iterable[Any]"  [arg-type]
src/fromager/hooks.py:9: error: Need type annotation for "_mgrs" (hint: "_mgrs: dict[<type>, <type>] = ...")  [var-annotated]
src/fromager/dependencies.py:33: error: Incompatible return value type (got "set[Requirement] | None", expected "set[Requirement]")  [return-value]
src/fromager/dependencies.py:60: error: Argument 3 to "evaluate_marker" has incompatible type "set[str]"; expected "dict[Any, Any] | None"  [arg-type]
src/fromager/dependencies.py:90: error: Incompatible return value type (got "set[Requirement] | None", expected "set[Requirement]")  [return-value]
src/fromager/dependencies.py:136: error: Incompatible return value type (got "set[Requirement] | None", expected "set[Requirement]")  [return-value]
src/fromager/dependencies.py:172: error: Argument 1 to "Wheel" has incompatible type "Path"; expected "str"  [arg-type]
src/fromager/dependencies.py:173: error: Argument 2 to "_filter_requirements" has incompatible type "Sequence[str]"; expected "Iterable[Requirement]"  [arg-type]
src/fromager/dependencies.py:202: error: Argument 3 to "evaluate_marker" has incompatible type "set[str]"; expected "dict[Any, Any] | None"  [arg-type]
src/fromager/sources.py:38: error: Function "downloader" could always be true in boolean context  [truthy-function]
src/fromager/sources.py:105: error: Need type annotation for "rslvr"  [var-annotated]
src/fromager/sources.py:120: error: Incompatible return value type (got "tuple[None, None]", expected "tuple[str, str]")  [return-value]
src/fromager/sources.py:164: error: Incompatible return value type (got "tuple[str, str, Any | str]", expected "tuple[Path, str, str]")  [return-value]
src/fromager/sources.py:186: error: Incompatible return value type (got "Path", expected "str")  [return-value]
src/fromager/sdist.py:130: error: Argument 2 to "has_been_seen" of "WorkContext" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:135: error: Argument 2 to "mark_as_seen" of "WorkContext" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:152: error: Argument 4 to "prepare_source" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:177: error: Argument "version" to "add_to_build_order" of "WorkContext" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:189: error: Incompatible types in assignment (expression has type "Path | None", variable has type "Path")  [assignment]
src/fromager/sdist.py:200: error: Item "None" of "Path | None" has no attribute "parent"  [union-attr]
src/fromager/sdist.py:213: error: Argument "version" to "build_sdist" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:214: error: Argument "sdist_root_dir" to "build_sdist" has incompatible type "Path | None"; expected "Path"  [arg-type]
src/fromager/sdist.py:228: error: Argument "sdist_root_dir" to "build_wheel" has incompatible type "Path | None"; expected "Path"  [arg-type]
src/fromager/sdist.py:229: error: Argument "version" to "build_wheel" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:235: error: Item "None" of "Path | None" has no attribute "name"  [union-attr]
src/fromager/sdist.py:414: error: Argument 4 to "_maybe_install" has incompatible type "None"; expected "str"  [arg-type]
src/fromager/sdist.py:436: error: Argument 4 to "_maybe_install" has incompatible type "None"; expected "str"  [arg-type]
src/fromager/sdist.py:455: error: Argument 4 to "_maybe_install" has incompatible type "None"; expected "str"  [arg-type]
src/fromager/sdist.py:485: error: Argument 3 to "MissingDependency" has incompatible type "str | Any"; expected "Requirement"  [arg-type]
src/fromager/commands/step.py:64: error: Argument 3 to "find_sdist" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/step.py:66: error: Need type annotation for "dir_contents" (hint: "dir_contents: list[<type>] = ...")  [var-annotated]
src/fromager/commands/step.py:97: error: Argument 3 to "_find_source_root_dir" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/step.py:98: error: Argument 3 to "BuildEnvironment" has incompatible type "None"; expected "Iterable[Requirement]"  [arg-type]
src/fromager/commands/step.py:141: error: Argument 3 to "_find_source_root_dir" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/step.py:162: error: Argument 3 to "_find_source_root_dir" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/step.py:163: error: Argument 3 to "BuildEnvironment" has incompatible type "None"; expected "Iterable[Requirement]"  [arg-type]
src/fromager/commands/build.py:110: error: Argument 3 to "BuildEnvironment" has incompatible type "None"; expected "Iterable[Requirement]"  [arg-type]
src/fromager/commands/build.py:134: error: Argument "dist_version" to "run_post_build_hooks" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/build.py:136: error: Argument "wheel_filename" to "run_post_build_hooks" has incompatible type "Path | None"; expected "Path"  [arg-type]
src/fromager/commands/build.py:139: error: Incompatible return value type (got "Path | None", expected "Path")  [return-value]
src/fromager/commands/bootstrap.py:17: error: Need type annotation for "to_build" (hint: "to_build: list[<type>] = ...")  [var-annotated]
src/fromager/commands/bootstrap.py:62: error: Incompatible types in assignment (expression has type "str", variable has type "list[str]")  [assignment]
src/fromager/commands/bootstrap.py:63: error: Argument 1 to "Requirement" has incompatible type "list[str]"; expected "str"  [arg-type]
Found 70 errors in 16 files (checked 31 source files)
mypy: exit 1 (2.96 seconds) /home/sbapna/projects/fromager> mypy -p fromager pid=186029
  mypy: FAIL code 1 (12.87=setup[9.91]+cmd[2.96] seconds)
  evaluation failed :( (12.90 seconds)

A stricter check shows 206 errors

src/fromager/extras_provider.py:24: error: Missing type parameters for generic type "AbstractProvider"  [type-arg]
src/fromager/extras_provider.py:27: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/extras_provider.py:34: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/extras_provider.py:41: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/extras_provider.py:43: error: Call to untyped function "get_extras_for" in typed context  [no-untyped-call]
src/fromager/extras_provider.py:49: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/extras_provider.py:52: error: Call to untyped function "get_base_requirement" in typed context  [no-untyped-call]
src/fromager/extras_provider.py:53: error: "Iterable[Any]" has no attribute "append"  [attr-defined]
src/fromager/external_commands.py:13: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/tarballs.py:25: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:36: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:44: error: Missing type parameters for generic type "typing.Callable"  [type-arg]
src/fromager/overrides.py:44: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:44: error: Function is missing a type annotation for one or more arguments  [no-untyped-def]
src/fromager/overrides.py:46: error: Function "fn" could always be true in boolean context  [truthy-function]
src/fromager/overrides.py:51: error: Missing type parameters for generic type "typing.Callable"  [type-arg]
src/fromager/overrides.py:51: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:51: error: Function is missing a type annotation for one or more arguments  [no-untyped-def]
src/fromager/overrides.py:62: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:62: note: Use "-> None" if function does not return a value
src/fromager/overrides.py:85: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/overrides.py:129: error: Missing type parameters for generic type "typing.Callable"  [type-arg]
src/fromager/overrides.py:145: error: Incompatible return value type (got "None", expected "Callable[..., Any]")  [return-value]
src/fromager/overrides.py:148: error: Incompatible return value type (got "None", expected "Callable[..., Any]")  [return-value]
src/fromager/overrides.py:150: error: Returning Any from function declared to return "Callable[..., Any]"  [no-any-return]
src/fromager/overrides.py:153: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:161: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:169: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:188: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:192: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/overrides.py:202: error: Call to untyped function "patched_projects" in typed context  [no-untyped-call]
src/fromager/overrides.py:203: error: Call to untyped function "patched_projects_legacy" in typed context  [no-untyped-call]
src/fromager/overrides.py:204: error: Call to untyped function "env_projects" in typed context  [no-untyped-call]
src/fromager/overrides.py:205: error: Call to untyped function "projects_with_predefined_download_source" in typed context  [no-untyped-call]
src/fromager/settings.py:12: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/settings.py:46: error: Argument 1 to "_parse" has incompatible type "TextIOWrapper"; expected "str"  [arg-type]
src/fromager/commands/canonicalize.py:8: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/vendor_rust.py:42: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/vendor_rust.py:68: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/vendor_rust.py:122: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/vendor_rust.py:122: note: Use "-> None" if function does not return a value
src/fromager/vendor_rust.py:149: error: Missing positional argument "project_dir" in call to "vendor_rust"  [call-arg]
src/fromager/vendor_rust.py:156: error: Call to untyped function "test" in typed context  [no-untyped-call]
src/fromager/requirements_file.py:29: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/requirements_file.py:38: error: List comprehension has incompatible type List[dict[str, object]]; expected List[Environment]  [misc]
src/fromager/requirements_file.py:41: error: Argument 1 to "evaluate" of "Marker" has incompatible type "Environment"; expected "dict[str, str] | None"  [arg-type]
src/fromager/finders.py:33: error: Function "sdist_name_func" could always be true in boolean context  [truthy-function]
src/fromager/finders.py:40: error: Returning Any from function declared to return "Path | None"  [no-any-return]
src/fromager/finders.py:123: error: Function "sdir_name_func" could always be true in boolean context  [truthy-function]
src/fromager/finders.py:130: error: Returning Any from function declared to return "Path | None"  [no-any-return]
src/fromager/constraints.py:19: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/constraints.py:22: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/constraints.py:54: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/constraints.py:54: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/constraints.py:62: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/constraints.py:64: error: Argument 1 to "_organize_constraints" has incompatible type "list[Any]"; expected "dict[Any, Any]"  [arg-type]
src/fromager/context.py:54: error: Need type annotation for "_build_stack" (hint: "_build_stack: list[<type>] = ...")  [var-annotated]
src/fromager/context.py:55: error: Need type annotation for "_build_requirements" (hint: "_build_requirements: set[<type>] = ...")  [var-annotated]
src/fromager/context.py:62: error: Need type annotation for "_seen_requirements" (hint: "_seen_requirements: set[<type>] = ...")  [var-annotated]
src/fromager/context.py:69: error: List item 1 has incompatible type "str | None"; expected "str"  [list-item]
src/fromager/context.py:72: error: Missing type parameters for generic type "tuple"  [type-arg]
src/fromager/context.py:75: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/context.py:80: error: Missing type parameters for generic type "tuple"  [type-arg]
src/fromager/context.py:81: error: Incompatible return value type (got "bool", expected "tuple[Any, ...]")  [return-value]
src/fromager/context.py:83: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/context.py:88: error: Missing type parameters for generic type "list"  [type-arg]
src/fromager/context.py:125: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/context.py:125: note: Use "-> None" if function does not return a value
src/fromager/commands/build_order.py:15: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/build_order.py:15: note: Use "-> None" if function does not return a value
src/fromager/commands/build_order.py:28: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/build_order.py:71: error: Incompatible types in assignment (expression has type "TextIO | Any", variable has type "TextIOWrapper")  [assignment]
src/fromager/commands/build_order.py:92: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/build_order.py:102: error: Need type annotation for "dist_to_input_file"  [var-annotated]
src/fromager/commands/build_order.py:113: error: Incompatible types in assignment (expression has type "TextIO | Any", variable has type "TextIOWrapper")  [assignment]
src/fromager/commands/build_order.py:146: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/build_order.py:153: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/commands/build_order.py:160: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/commands/build_order.py:168: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/commands/build_order.py:169: error: Call to untyped function "new_node" in typed context  [no-untyped-call]
src/fromager/commands/build_order.py:178: error: Need type annotation for "nodes" (hint: "nodes: dict[<type>, <type>] = ...")  [var-annotated]
src/fromager/commands/build_order.py:186: error: Call to untyped function "update_node" in typed context  [no-untyped-call]
src/fromager/commands/build_order.py:187: error: Call to untyped function "fmt_req" in typed context  [no-untyped-call]
src/fromager/commands/build_order.py:200: error: Call to untyped function "update_node" in typed context  [no-untyped-call]
src/fromager/commands/build_order.py:200: error: Call to untyped function "fmt_req" in typed context  [no-untyped-call]
src/fromager/commands/build_order.py:201: error: Call to untyped function "update_node" in typed context  [no-untyped-call]
src/fromager/commands/build_order.py:201: error: Call to untyped function "fmt_req" in typed context  [no-untyped-call]
src/fromager/commands/build_order.py:213: error: Incompatible types in assignment (expression has type "TextIO | Any", variable has type "TextIOWrapper")  [assignment]
src/fromager/commands/build_order.py:282: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/build_order.py:323: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/wheels.py:30: error: Call to untyped function "_createenv" in typed context  [no-untyped-call]
src/fromager/wheels.py:36: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/wheels.py:36: note: Use "-> None" if function does not return a value
src/fromager/wheels.py:42: error: List item 3 has incompatible type "Path"; expected "str"  [list-item]
src/fromager/wheels.py:54: error: Argument 1 to "run" has incompatible type "list[object]"; expected "list[str]"  [arg-type]
src/fromager/wheels.py:68: error: Argument "cwd" to "run" has incompatible type "Path"; expected "str | None"  [arg-type]
src/fromager/wheels.py:120: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/server.py:14: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/server.py:14: error: Function is missing a type annotation for one or more arguments  [no-untyped-def]
src/fromager/server.py:18: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/server.py:33: error: Argument "directory" to "LoggingHTTPRequestHandler" has incompatible type "Path"; expected "str | None"  [arg-type]
src/fromager/server.py:46: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/server.py:57: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/resolver.py:46: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/resolver.py:64: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/resolver.py:66: error: Incompatible types in assignment (expression has type "EmailMessage", variable has type "None")  [assignment]
src/fromager/resolver.py:66: error: Argument 1 to "get_metadata_for_wheel" has incompatible type "str | None"; expected "str"  [arg-type]
src/fromager/resolver.py:71: error: Returning Any from function declared to return "bool"  [no-any-return]
src/fromager/resolver.py:89: error: Incompatible types in assignment (expression has type "list[Requirement]", variable has type "None")  [assignment]
src/fromager/resolver.py:90: error: Incompatible return value type (got "None", expected "list[Requirement]")  [return-value]
src/fromager/resolver.py:137: error: Need type annotation for "tags" (hint: "tags: set[<type>] = ...")  [var-annotated]
src/fromager/resolver.py:140: error: Incompatible types in assignment (expression has type "frozenset[Tag]", variable has type "set[Any]")  [assignment]
src/fromager/resolver.py:167: error: Argument "extras" to "Candidate" has incompatible type "tuple[str]"; expected "dict[Any, Any] | None"  [arg-type]
src/fromager/resolver.py:180: error: Incompatible return value type (got "Message[str, str]", expected "EmailMessage")  [return-value]
src/fromager/resolver.py:208: error: Argument 1 to "sorted" has incompatible type "set[str] | dict[Any, Any] | None"; expected "Iterable[Any]"  [arg-type]
src/fromager/resolver.py:213: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/resolver.py:213: error: Signature of "get_preference" incompatible with supertype "AbstractProvider"  [override]
src/fromager/resolver.py:213: note:      Superclass:
src/fromager/resolver.py:213: note:          def get_preference(self, identifier: Any, resolutions: Mapping[Any, Any], candidates: Mapping[Any, Iterator[Any]], information: Mapping[Any, Iterator[RequirementInformation[Any, Any]]], backtrack_causes: Sequence[RequirementInformation[Any, Any]]) -> Preference
src/fromager/resolver.py:213: note:      Subclass:
src/fromager/resolver.py:213: note:          def get_preference(self, identifier: Any, resolutions: Any, candidates: Any, information: Any, **kwds: Any) -> Any
src/fromager/resolver.py:216: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/resolver.py:225: error: Argument 2 to "get_project_from_pypi" has incompatible type "set[Never]"; expected "tuple[str]"  [arg-type]
src/fromager/resolver.py:253: error: Missing type parameters for generic type "list"  [type-arg]
src/fromager/resolver.py:277: error: Argument 1 to "sorted" has incompatible type "set[str] | dict[Any, Any] | None"; expected "Iterable[Any]"  [arg-type]
src/fromager/resolver.py:282: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/resolver.py:282: error: Signature of "get_preference" incompatible with supertype "AbstractProvider"  [override]
src/fromager/resolver.py:282: note:      Superclass:
src/fromager/resolver.py:282: note:          def get_preference(self, identifier: Any, resolutions: Mapping[Any, Any], candidates: Mapping[Any, Iterator[Any]], information: Mapping[Any, Iterator[RequirementInformation[Any, Any]]], backtrack_causes: Sequence[RequirementInformation[Any, Any]]) -> Preference
src/fromager/resolver.py:282: note:      Subclass:
src/fromager/resolver.py:282: note:          def get_preference(self, identifier: Any, resolutions: Any, candidates: Any, information: Any, **kwds: Any) -> Any
src/fromager/resolver.py:285: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/hooks.py:9: error: Need type annotation for "_mgrs" (hint: "_mgrs: dict[<type>, <type>] = ...")  [var-annotated]
src/fromager/hooks.py:28: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/hooks.py:36: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/dependencies.py:33: error: Incompatible return value type (got "set[Requirement] | None", expected "set[Requirement]")  [return-value]
src/fromager/dependencies.py:60: error: Argument 3 to "evaluate_marker" has incompatible type "set[str]"; expected "dict[Any, Any] | None"  [arg-type]
src/fromager/dependencies.py:73: error: Returning Any from function declared to return "Iterable[Requirement]"  [no-any-return]
src/fromager/dependencies.py:90: error: Incompatible return value type (got "set[Requirement] | None", expected "set[Requirement]")  [return-value]
src/fromager/dependencies.py:119: error: Returning Any from function declared to return "set[Requirement]"  [no-any-return]
src/fromager/dependencies.py:136: error: Incompatible return value type (got "set[Requirement] | None", expected "set[Requirement]")  [return-value]
src/fromager/dependencies.py:165: error: Returning Any from function declared to return "set[Requirement]"  [no-any-return]
src/fromager/dependencies.py:172: error: Argument 1 to "Wheel" has incompatible type "Path"; expected "str"  [arg-type]
src/fromager/dependencies.py:173: error: Argument 2 to "_filter_requirements" has incompatible type "Sequence[str]"; expected "Iterable[Requirement]"  [arg-type]
src/fromager/dependencies.py:202: error: Argument 3 to "evaluate_marker" has incompatible type "set[str]"; expected "dict[Any, Any] | None"  [arg-type]
src/fromager/dependencies.py:207: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/dependencies.py:222: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/dependencies.py:239: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/dependencies.py:243: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/dependencies.py:262: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/list_overrides.py:8: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/sources.py:38: error: Function "downloader" could always be true in boolean context  [truthy-function]
src/fromager/sources.py:105: error: Need type annotation for "rslvr"  [var-annotated]
src/fromager/sources.py:120: error: Incompatible return value type (got "tuple[None, None]", expected "tuple[str, str]")  [return-value]
src/fromager/sources.py:164: error: Incompatible return value type (got "tuple[str, str, Any | str]", expected "tuple[Path, str, str]")  [return-value]
src/fromager/sources.py:186: error: Incompatible return value type (got "Path", expected "str")  [return-value]
src/fromager/sources.py:212: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/sources.py:236: error: Missing type parameters for generic type "typing.Callable"  [type-arg]
src/fromager/sources.py:304: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/sources.py:307: error: Returning Any from function declared to return "dict[Any, Any]"  [no-any-return]
src/fromager/sources.py:329: error: Returning Any from function declared to return "Path"  [no-any-return]
src/fromager/sources.py:366: error: Returning Any from function declared to return "Path"  [no-any-return]
src/fromager/sources.py:371: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/sources.py:399: error: Missing type parameters for generic type "dict"  [type-arg]
src/fromager/sources.py:412: error: Returning Any from function declared to return "Path"  [no-any-return]
src/fromager/commands/server.py:23: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/sdist.py:67: error: Missing type parameters for generic type "list"  [type-arg]
src/fromager/sdist.py:130: error: Argument 2 to "has_been_seen" of "WorkContext" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:135: error: Argument 2 to "mark_as_seen" of "WorkContext" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:152: error: Argument 4 to "prepare_source" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:177: error: Argument "version" to "add_to_build_order" of "WorkContext" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:189: error: Incompatible types in assignment (expression has type "Path | None", variable has type "Path")  [assignment]
src/fromager/sdist.py:200: error: Item "None" of "Path | None" has no attribute "parent"  [union-attr]
src/fromager/sdist.py:213: error: Argument "version" to "build_sdist" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:214: error: Argument "sdist_root_dir" to "build_sdist" has incompatible type "Path | None"; expected "Path"  [arg-type]
src/fromager/sdist.py:228: error: Argument "sdist_root_dir" to "build_wheel" has incompatible type "Path | None"; expected "Path"  [arg-type]
src/fromager/sdist.py:229: error: Argument "version" to "build_wheel" has incompatible type "str"; expected "Version"  [arg-type]
src/fromager/sdist.py:235: error: Item "None" of "Path | None" has no attribute "name"  [union-attr]
src/fromager/sdist.py:285: error: Call to untyped function "_download_wheel_check" in typed context  [no-untyped-call]
src/fromager/sdist.py:295: error: Function is missing a type annotation  [no-untyped-def]
src/fromager/sdist.py:331: error: Missing type parameters for generic type "list"  [type-arg]
src/fromager/sdist.py:355: error: Missing type parameters for generic type "list"  [type-arg]
src/fromager/sdist.py:379: error: Missing type parameters for generic type "list"  [type-arg]
src/fromager/sdist.py:414: error: Argument 4 to "_maybe_install" has incompatible type "None"; expected "str"  [arg-type]
src/fromager/sdist.py:436: error: Argument 4 to "_maybe_install" has incompatible type "None"; expected "str"  [arg-type]
src/fromager/sdist.py:455: error: Argument 4 to "_maybe_install" has incompatible type "None"; expected "str"  [arg-type]
src/fromager/sdist.py:485: error: Argument 3 to "MissingDependency" has incompatible type "str | Any"; expected "Requirement"  [arg-type]
src/fromager/sdist.py:492: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/sdist.py:517: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/step.py:14: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/step.py:14: note: Use "-> None" if function does not return a value
src/fromager/commands/step.py:24: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/step.py:50: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/step.py:64: error: Argument 3 to "find_sdist" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/step.py:66: error: Need type annotation for "dir_contents" (hint: "dir_contents: list[<type>] = ...")  [var-annotated]
src/fromager/commands/step.py:81: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/step.py:97: error: Argument 3 to "_find_source_root_dir" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/step.py:98: error: Argument 3 to "BuildEnvironment" has incompatible type "None"; expected "Iterable[Requirement]"  [arg-type]
src/fromager/commands/step.py:127: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/step.py:141: error: Argument 3 to "_find_source_root_dir" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/step.py:149: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/step.py:162: error: Argument 3 to "_find_source_root_dir" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/step.py:163: error: Argument 3 to "BuildEnvironment" has incompatible type "None"; expected "Iterable[Requirement]"  [arg-type]
src/fromager/commands/download_sequence.py:22: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/build.py:19: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/build.py:54: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/build.py:110: error: Argument 3 to "BuildEnvironment" has incompatible type "None"; expected "Iterable[Requirement]"  [arg-type]
src/fromager/commands/build.py:134: error: Argument "dist_version" to "run_post_build_hooks" has incompatible type "Version"; expected "str"  [arg-type]
src/fromager/commands/build.py:136: error: Argument "wheel_filename" to "run_post_build_hooks" has incompatible type "Path | None"; expected "Path"  [arg-type]
src/fromager/commands/build.py:139: error: Incompatible return value type (got "Path | None", expected "Path")  [return-value]
src/fromager/commands/bootstrap.py:17: error: Need type annotation for "to_build" (hint: "to_build: list[<type>] = ...")  [var-annotated]
src/fromager/commands/bootstrap.py:38: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/commands/bootstrap.py:62: error: Incompatible types in assignment (expression has type "str", variable has type "list[str]")  [assignment]
src/fromager/commands/bootstrap.py:63: error: Argument 1 to "Requirement" has incompatible type "list[str]"; expected "str"  [arg-type]
src/fromager/__main__.py:101: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/__main__.py:101: error: Function is missing a type annotation for one or more arguments  [no-untyped-def]
src/fromager/__main__.py:151: error: Call to untyped function "log_overrides" in typed context  [no-untyped-call]
src/fromager/__main__.py:166: error: Call to untyped function "setup" in typed context  [no-untyped-call]
src/fromager/__main__.py:174: error: Function is missing a return type annotation  [no-untyped-def]
src/fromager/__main__.py:174: note: Use "-> None" if function does not return a value
src/fromager/__main__.py:185: error: Call to untyped function "invoke_main" in typed context  [no-untyped-call]
Found 206 errors in 26 files (checked 31 source files)

dhellmann commented 2 months ago

@tiran has started trying to fix this in #110 , but I think the code has changed a lot since then.

We may want to try to improve things in small PRs instead of trying to land all of the fixes at once.

shubhbapna commented 2 months ago

@tiran has started trying to fix this in #110 , but I think the code has changed a lot since then.

Yep have already discussed this with him and he said I can take over.

We may want to try to improve things in small PRs instead of trying to land all of the fixes at once.

Yes, I am tracking each file and will probably have separate PRs for files with too many fixes

rd4398 commented 2 months ago

@shubhbapna let me know if you need help with this / want me to pick some files (since the list is large)

shubhbapna commented 2 months ago

@shubhbapna let me know if you need help with this / want me to pick some files (since the list is large)

Sure feel free to take any file

shubhbapna commented 2 months ago

@dhellmann why do we have this constraint on pyproject_hooks?

pyproject_hooks>=1.0.0,!=1.1.0

The 1.1.0 version comes with typings

dhellmann commented 2 months ago

@dhellmann why do we have this constraint on pyproject_hooks?
pyproject_hooks>=1.0.0,!=1.1.0
The 1.1.0 version comes with typings

I probably copied that blindly from somewhere else. I don't know of any reason 1.1.0 wouldn't work. Remove the constraint and see what happens?

shubhbapna commented 2 months ago

@dhellmann why do we have this constraint on pyproject_hooks?
pyproject_hooks>=1.0.0,!=1.1.0
The 1.1.0 version comes with typings
I probably copied that blindly from somewhere else. I don't know of any reason 1.1.0 wouldn't work. Remove the constraint and see what happens?

Figured out why we need that constraint. 1.1.0 causes a regression: https://github.com/pypa/pyproject-hooks/issues/192 hence on upgrading our pipelines fail: https://github.com/python-wheel-build/fromager/actions/runs/10080115233/job/27869097494#step:6:86

dhellmann commented 2 months ago

@dhellmann why do we have this constraint on pyproject_hooks?
pyproject_hooks>=1.0.0,!=1.1.0
The 1.1.0 version comes with typings
I probably copied that blindly from somewhere else. I don't know of any reason 1.1.0 wouldn't work. Remove the constraint and see what happens?
Figured out why we need that constraint. 1.1.0 causes a regression: pypa/pyproject-hooks#192 hence on upgrading our pipelines fail: https://github.com/python-wheel-build/fromager/actions/runs/10080115233/job/27869097494#step:6:86

We should have a comment in our requirements list with a link to that bug so we can remember why the exclusion is there next time someone asks. :-)

tiran commented 2 months ago

PR #282 enforces MyPy checks and ignores errors in files with known issues.

python-wheel-build / fromager

tracker: enforce typings and enable mypy checks in github actions #226