Mypy pre-commit and editable install

[lucascolley]

Hey @jorenham, apologies for the very random question that seems google-able but I'm a bit stuck and figured you know what you are talking about when it comes to typing :)

At https://github.com/lucascolley/array-api-extra/blob/97a792a06bab290abe895eed48e4f8dcc2c0284b/tests/test_funcs.py#L3-L4 I import from some optional test dependencies including numpy. I have Mypy in my pre-commit hook. If I am developing in an environment without my optional test dependencies, Mypy complains about missing imports.

Is the only solution to have all optional dependencies (from which I import) installed when running Mypy?

[lucascolley]

Huh, I just found this:

By default, mypy will run with mypy --ignore-missing-imports, pre-commit runs mypy from an isolated virtualenv so it won't have access to those. To change the arguments, override the args as follows:

    hooks:
    -   id: mypy
        args: [--strict, --ignore-missing-imports]

Because pre-commit runs mypy from an isolated virtualenv (without your dependencies) you may also find it useful to add the typed dependencies to additional_dependencies so mypy can better perform dynamic analysis:

    hooks:
    -   id: mypy
        additional_dependencies: [tokenize-rt==3.2.0]

Note that using the --install-types is problematic. Mutating the pre-commit environment at runtime breaks cache and will break parallel builds.

[jorenham]

Static type-checkers like mypy require the type annotations, which can be either inlined in in the .py files, or provided externally as .pyi stubs. NumPy is an example that uses stubs. But unlike e.g. pandas, it doesn't package them separately (anymore).

So to answer you question:

Is the only solution to have all optional dependencies (from which I import) installed when running Mypy?

Yes, when running mypy on your project, it'll need to have numpy available within the same environment.

[jorenham]

In case of pre-commit, you can also configure it to run mypy within your existing environment.

For an example in a poetry-based project on mine, see e.g.: https://github.com/jorenham/optype/blob/master/.pre-commit-config.yaml#L82-L86

and for a uv-based project, see https://github.com/jorenham/mainpy/pull/51/files#diff-63a9c44a44acf85fea213a857769990937107cf072831e1a26808cfde9d096b9R65-R70

[lucascolley]

Okay great, looks like

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index bf0885c..97cb4c9 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -56,6 +56,8 @@ repos:
         args: []
         additional_dependencies:
           - pytest
+          - numpy
+          - array-api-strict

does the job (modulo https://github.com/data-apis/array-api-strict/issues/6#issuecomment-1944036046). I guess it was google-able 🙃 thanks for the pointers!

[lucascolley]

Hmm, I'm now hitting an issue with importing from this package as an editable install. pre-commit install && pre-commit run -v --all-files --show-diff-on-failure passes, but when actually committing pre-commit throws:

tests/test_funcs.py:6: error: Cannot find implementation or library stub for module named "array_api_extra"  [import-not-found]

with the hook

# start templated
INSTALL_PYTHON=/Users/lucascolley/.pixi/envs/pre-commit/bin/python3.12
ARGS=(hook-impl --config=.pre-commit-config.yaml --hook-type=pre-commit)
# end templated

HERE="$(cd "$(dirname "$0")" && pwd)"
ARGS+=(--hook-dir "$HERE" -- "$@")

if [ -x "$INSTALL_PYTHON" ]; then
    exec "$INSTALL_PYTHON" -mpre_commit "${ARGS[@]}"
elif command -v pre-commit > /dev/null; then
    exec pre-commit "${ARGS[@]}"
else
    echo '`pre-commit` not found.  Did you forget to activate your virtualenv?' 1>&2
    exit 1
fi

they should both be using the same config file, so why does one fail but the other pass?

[lucascolley]

I've fixed this for now by separating mypy from pre-commit 👍 (that seems to be the recommendation from mypy devs at least)