python-attrs / attrs

Python Classes Without Boilerplate
https://www.attrs.org/
MIT License
5.28k stars 369 forks source link

Mypy error "Cannot determine __init__ type from converter" when using type checking, overloads and converters #897

Open dmaasland opened 2 years ago

dmaasland commented 2 years ago

Hello,

I'm not sure how to explain this properly but I'll do my best. Imagine a project with the following layout:

test
├── __init__.py
├── py.typed
├── test.py
└── test_functions.py

The __init__.py and py.typed files are both empty. The other two files contain the following test code:

test.py

from datetime import datetime
from typing import Optional
from attrs import define, field

from .test_functions import to_datetime

@define
class Test:
    test_string: str
    test_date: Optional[datetime] = field(converter=to_datetime, repr=str)

test_functions.py

from datetime import datetime
from typing import overload
from typing import Optional

@overload
def to_datetime(value: None) -> None:
    pass

@overload
def to_datetime(value: str) -> datetime:
    pass

def to_datetime(value: Optional[str]) -> Optional[datetime]:
    if value is None:
        return None

    retval = datetime.strptime(value, "%Y-%m-%dT%H:%M:%S%z")
    return retval

This all fine and passes mypy checks:

(.venv) vscode ➜ /workspaces/test $ mypy test
Success: no issues found in 3 source files
(.venv) vscode ➜ /workspaces/test $ 

However, if I move the functions from the test_functions.py file over to the test.py like so:

test.py

from datetime import datetime
from typing import Optional, overload
from attrs import define, field

# from .test_functions import to_datetime

@overload
def to_datetime(value: None) -> None:
    pass

@overload
def to_datetime(value: str) -> datetime:
    pass

def to_datetime(value: Optional[str]) -> Optional[datetime]:
    if value is None:
        return None

    retval = datetime.strptime(value, "%Y-%m-%dT%H:%M:%S%z")
    return retval

@define
class Test:
    test_string: str
    test_date: Optional[datetime] = field(converter=to_datetime, repr=str)

mypy will fail with the following error: test/test.py:29: error: Cannot determine __init__ type from converter

(.venv) vscode ➜ /workspaces/test $ mypy test
test/test.py:29: error: Cannot determine __init__ type from converter
Found 1 error in 1 file (checked 3 source files)
(.venv) vscode ➜ /workspaces/test $ 

I've been scratching my head for hours about this, but I cannot seem to find a valid reason for it. I really don't want some functions in a seperate file as that will cause an even harder to solve import loop :).

Is this a bug or something I'm not properly understanding / doing wrong?

Thanks!

PS: attrs version 21.3.0, python version 3.7.12.

Tinche commented 2 years ago

Interesting, looks like a Mypy bug to me. Possibly in the attrs Mypy plugin.

Using reveal_type, the output is identical.

Same file:

note: Revealed type is "Overload(def (value: None), def (value: builtins.str) -> datetime.datetime)"
error: Cannot determine __init__ type from converter

Different files:

note: Revealed type is "Overload(def (value: None), def (value: builtins.str) -> datetime.datetime)"

Can also confirm it's not an attr/attrs issue, the error is present in both scenarios.

euresti commented 2 years ago

The processing of converter is pretty complex and I don't think I tested every scenario. My guess is this happens because of the multi-pass process that mypy uses but I haven't investigated.

Tinche commented 2 years ago

Opened a PR over at Mypy for this (GitHub link right above). @euresti Any chance of a review over there?