typeddjango / pytest-mypy-plugins

pytest plugin for testing mypy types, stubs, and plugins
https://pypi.org/project/pytest-mypy-plugins/
MIT License
100 stars 26 forks source link

Adjust test-mypy-config.yml to test with more stable mypy options #137

Closed antecrescent closed 6 months ago

antecrescent commented 6 months ago

Since mypy 1.6.0+, the test pytest_mypy_plugins/tests/test-mypy-config fails with:

$ pytest pytest_mypy_plugins/tests/test-mypy-config.yml -vvv
============================================================================== test session starts ==============================================================================
platform linux -- Python 3.11.5, pytest-7.4.3, pluggy-1.3.0 -- /home/delgan/programming/pytest-mypy-plugins/env/bin/python
cachedir: .pytest_cache
rootdir: /home/delgan/programming/pytest-mypy-plugins
configfile: pyproject.toml
plugins: mypy-plugins-3.0.0
collected 1 item                                                                                                                                                                

pytest_mypy_plugins/tests/test-mypy-config.yml::custom_mypy_config_strict_optional_true_set FAILED

=================================================================================== FAILURES ====================================================================================
__________________________________________________________________ custom_mypy_config_strict_optional_true_set __________________________________________________________________
/home/delgan/programming/pytest-mypy-plugins/pytest_mypy_plugins/tests/test-mypy-config.yml:4: 
E   pytest_mypy_plugins.utils.TypecheckAssertionError: Output is not expected: 
E   Actual:
E     ../../home/delgan/programming/pytest-mypy-plugins/env/lib/python3.11/site-packages/mypy/typeshed/stdlib/builtins.pyi:154: error: Overloaded function signature 2 will never be matched: signature 1's parameter type(s) are the same or broader  [misc] (diff)
E   Expected:
E     (empty)
============================================================================ short test summary info ============================================================================
FAILED pytest_mypy_plugins/tests/test-mypy-config.yml::custom_mypy_config_strict_optional_true_set - 
=============================================================================== 1 failed in 3.14s ===============================================================================

Running mypy with --no-strict-optional is discouraged and not much tested. This issue can be worked around by appending --mypy-only-local-stub to all pytest calls. Original thread: https://github.com/typeddjango/pytest-mypy-plugins/pull/135/files#r1382473671 Upstream Gentoo Linux report: https://bugs.gentoo.org/921901

To fix this issue, I propose to test inline mypy configuration with the following test case instead:

# Also used in `test_explicit_configs.py`

- case: custom_mypy_config_disallow_any_explicit_set
  expect_fail: yes
  main: |
    from typing import Any
    a: Any = None  # should raise an error
  mypy_config: |
    disallow_any_explicit = true

If this is a sane approach, I'm happy to open a PR :)

sobolevn commented 6 months ago

Let's try this.