Open KotlinIsland opened 1 month ago
I am seeing what might be a related issue. For me it doesn't result in a crash, but I am getting errors for rows which aren't in my file. I'm getting a bunch of errors like:
my_file.py:119: error: X | Y syntax for unions requires Python 3.10 [syntax]
my_file.py:120: error: X | Y syntax for unions requires Python 3.10 [syntax]
my_file.py:121: error: X | Y syntax for unions requires Python 3.10 [syntax]
my_file.py:122: error: X | Y syntax for unions requires Python 3.10 [syntax]
my_file.py:123: error: X | Y syntax for unions requires Python 3.10 [syntax]
...
My file my_file.py
has only 30 lines.
It happens on three different files in my codebase, and what they all have in common is that they all have the import from pydantic_settings import BaseSettings
.
I'm using python 3.8, and in the file in pydantic-settings
where BaseSettings
is defined, lines 129-136 are the parameter list for BaseSettings.__init__()
, and they are all type-annnotated as something | None
.
It looks like somehow mypy thinks those lines are in my source file?
Another strange detail: on the second run (if I already have a .mypy_cache/
) it doesn't fail. But if I delete the cache and run again the errors come back.
I can reproduce it like this:
from pydantic_settings import BaseSettings
class Foo(BaseSettings):
pass
$ mypy test.py
test.py:119: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:120: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:121: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:122: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:123: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:124: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:125: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:126: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:127: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:128: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:129: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:130: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:131: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:132: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:133: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:134: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:135: error: X | Y syntax for unions requires Python 3.10 [syntax]
test.py:136: error: X | Y syntax for unions requires Python 3.10 [syntax]
Found 18 errors in 1 file (checked 1 source file)
environment: mypy 1.11.0 python 3.8.19 pydantic-settings 2.3.4 pydantic 2.8.2 pydantic mypy plugin enabled
It doesn't replicate with the mypy plugin disabled, but it doesn't replicate with mypy 1.10.1 either.
@brianmedigate add --pretty
you will probably crash
confirmed, you are correct. thanks
I have the same problem. Python 3.9.19, mypy 1.11.0
A file that imports from pydantic_settings gets a bunch of errors related to 3.10 union syntax. That syntax is used in pydantic-settings, but it is not used in my file. Also, the line numbers stated in the mypy output are greater than the number of lines in my file. It somehow seems to think some pydantic_settings file is my file.
If the mypy cache exists, the error does not show up. If I delete the cache and re-run, the errors are back.
Calling mypy with --pretty
changes the listed errors to a crash (otherwise it behaves the same with regards to crashing).
Traceback (most recent call last):
File "REDACTED/venv/bin/mypy", line 8, in <module>
sys.exit(console_entry())
File "REDACTED/venv/lib/python3.9/site-packages/mypy/__main__.py", line 15, in console_entry
main()
File "mypy/main.py", line 103, in main
File "mypy/main.py", line 187, in run_build
File "mypy/build.py", line 193, in build
File "mypy/build.py", line 268, in _build
File "mypy/build.py", line 2950, in dispatch
File "mypy/build.py", line 3348, in process_graph
File "mypy/build.py", line 3471, in process_stale_scc
File "mypy/errors.py", line 923, in file_messages
File "mypy/errors.py", line 883, in format_messages
IndexError: list index out of range
Will this problem be tracked in this issue? (It was not clear to me from the title that this was the same problem I had.)
@brianmedigate I am witnessing the behavior you described in files with BaseSettings
imported.
@KotlinIsland does your use case include pydantic? I wonder if we should cross-post this to the pydantic issues page?
Hitting the same issue, and I'm using Pydantic's BaseSettings
where I'm hitting this.
Flagged it to pydantic-settings
here: https://github.com/pydantic/pydantic-settings/issues/349
Pydantic dev here — I'll note that I can reproduce the error with Python 3.9.19 and Mypy 1.11.0, but the issue goes away if I use python >=3.10 or mypy <=1.10.1.
I do think it has something to do with the fact that pydantic-settings
uses the X | Y
union syntax and from __future__ import annotations
, given the presumably spurious errors being produced without the --pretty
flag that reference this syntax, and the fact that upgrading past 3.9 fixes it.
Perhaps it is an issue with detecting the from __future__ import annotations
? (Technically we are using from __future__ import annotations as _annotations
at the top of the relevant file in pydantic-settings
.)
Thanks a bunch for the additional pointers here, @dmontagu.
@KotlinIsland does your use case include pydantic? I wonder if we should cross-post this to the pydantic issues page?
I found it from the primer output in a basedmypy pull request, the primer is much more comprehensive in basedmypy, so it hasn't been detected in mypy
Any updates here?
OK, I looked at it and in both cases it is a bug in the plugins, not sure why they are surfaced only now. For attrs
plugin I can just fix it, because it is hosted in mypy. For pydantic I am not sure what to do, the problem is that all those errors are coming from (aparrently generated) Foo.__init__
(in the example above). It looks like the plugin simply copies types from the __init__
of supertype including all metadata, like line/column number and whether it was created using new syntax, etc. I can try disabling this error for plugin generated methods, but no guarantees, ultimately plugin authors may need to fix the plugin.
Looking at commit history, the pydantic issue was likely triggered by https://github.com/python/mypy/pull/17371 cc @hauntsaninja The default copy behavior for union is now to carry the uses_pep604_syntax
flag, which is actually somewhat questionable.
@ilevkivskyi if you end up needing to modify the attrs plugin, I am happy to take responsibility (or at least to try to take responsibility 😅) for adapting the same fix to the pydantic plugin. I would just appreciate if you could share a link to any such PR on this issue (or vice versa).
To be clear, the pydantic plugin was adapted from the dataclasses plugin some time ago, and every so often I try to update it to reflect updates/improvements in the dataclasses plugin; if the dataclasses and/or attrs plugins now copy the "correct" subset of metadata data (instead of all supertype method metadata), I can try to modify the pydantic plugin to handle it correctly, it would just be helpful to have some kind of reference to a spot where this is being done "properly" from your perspective.
I made a change in basedmypy to prevent the crash if the requested source code doesn't exist
threading.py:316: error
, threading only has 43 lines in it.