Updates to DataFrame.__new__ causes issue with custom DataFrame Generic types

[cosmicBboy]

Describe the bug

Hi all, first off thanks for this effort! We use pandas-stubs in the pandera project for optional mypy type linting and it's been working great!

Recent changes (here) have caused issues in pandera type annotation, where we use a pandera SchemaModel in a subclassed DataFrame class like so:

class MySchema(pandera.SchemaModel): ...

# then you can initialize a validated dataframe like so:
vaildated_data = pandera.typing.DataFrame[MySchema](data)

However, this will raise a mypy error if pandas-stubs is installed:

Type application has too few types (2 expected)

To Reproduce

Consider this code, which subclasses dataframes :

# bar.py
from typing import Generic, TypeVar

import pandas as pd

T = TypeVar("T")

class MyDataFrame(pd.DataFrame, Generic[T]):
    ...

def func() -> MyDataFrame[int]:
    return MyDataFrame[int]({"foo": [1, 2, 3]})

Then using mypy:

mypy bar.py

# bar.py:14: error: Type application has too few types (2 expected)  [misc]
# Found 1 error in 1 file (checked 1 source file)

I've isolated the cause of this here, where removing the ListLike type annotations (including the one in the Iterable type) makes the error go away. I don't understand why the type annotations in the __new__ method would effect the type of the subclassed generic arguments 🤷‍♂️

Please complete the following information:

• OS: MacOS
• OS Version: 10.15.7
• python version: 3.9.12
• version of type checker: mypy==0.971
• version of installed pandas-stubs: 1.4.3.220815

Additional context

This issue doesn't show up in the previous version pandas-stubs==1.4.3.220807

[Dr-Irv]

Thanks for reporting this. Apparently you can't use a TypeVar as an argument in __new__() . So I created a Union instead, and now things work.

[twoertwein]

Might be good to report a minimal example to mypy, pyright doesn't have this issue. (But I'm surprised that pyright's reportInvalidTypeVarUse didn't complain about it.)

[Dr-Irv]

Might be good to report a minimal example to mypy, pyright doesn't have this issue. (But I'm surprised that pyright's reportInvalidTypeVarUse didn't complain about it.)

Added https://github.com/python/mypy/issues/13437

[cosmicBboy]

thanks for the quick response @Dr-Irv ! For the time being I've pinned pandas-stubs<=1.4.3.220807 in pandera for now, will unpin it in the next release once this change makes it into a new pandas-stubs release.

[Dr-Irv]

thanks for the quick response @Dr-Irv ! For the time being I've pinned pandas-stubs<=1.4.3.220807 in pandera for now, will unpin it in the next release once this change makes it into a new pandas-stubs release.

I've been pretty steady in doing releases each week, so next one should be on 8/21 or 8/22

[Dr-Irv]

@cosmicBboy Just released v1.4.3.220822 that includes the fix for this issue.