pandas-dev / pandas-stubs

Public type stubs for pandas
BSD 3-Clause "New" or "Revised" License
233 stars 123 forks source link

var-annotated error for Series and Index after upgrading to 2.0.3+ #826

Open amgcc opened 10 months ago

amgcc commented 10 months ago

Describe the bug Upgrading from 2.0.1 to 2.1.1 and now I get errors about Series and Index that aren't helpful. It should not be necessary to duplicate type hints on a local variable.

To Reproduce

  1. Provide a minimal runnable pandas example that is not properly checked by the stubs.

    idx0 = pd.Index()
    idx1: pd.Index = pd.Index()
    sr0 = pd.Series()
    sr1: pd.Series = pd.Series()
  2. Indicate which type checker you are using (mypy or pyright). mypy

  3. Show the error message received from that type checker while checking your example.

    error: Need type annotation for "idx0"  [var-annotated]
    error: Need type annotation for "sr0"  [var-annotated]

Please complete the following information:

twoertwein commented 10 months ago

I can re-produce the var-annotated error for pd.Index but not for pd.Series.

Revealed type is "pandas.core.indexes.base.Index[]" Revealed type is "pandas.core.series.Series[Any]"

Probably need to slightly adjust the Index overloads to return Index[Any] (or Index[S1]) in the last case?

Quick note: pandas 2.1.* requires python 3.9 or newer! This also means you are not getting the latest pandas-stubs releases (they also require 3.9)!

amgcc commented 10 months ago

I can re-produce the var-annotated error for pd.Index but not for pd.Series.

Revealed type is "pandas.core.indexes.base.Index[]" Revealed type is "pandas.core.series.Series[Any]"

Probably need to slightly adjust the Index overloads to return Index[Any] (or Index[S1]) in the last case?

Quick note: pandas 2.1.* requires python 3.9 or newer! This also means you are not getting the latest pandas-stubs releases (they also require 3.9)!

I am able to resolve 2.1.1 with Python 3.8 and while I plan to upgrade soon, it wouldn't help unless this gets fixed in the stubs library first. This issue affects pd.Series and not just pd.Index. It shouldn't matter but my version of pandas I am testing with is 2.0.3.

Additional migration issues for the pd.Series constructor involve passing a dictionary. It fails with a Timestamp or multi-index key whereas before it did not:

sr1 = pd.Series(
        {
            pd.Timestamp(2023, 1, 2): "b",
        }
    )

    sr2 = pd.Series(
        {
            ("a", "b"): "c",
        }
    )
test.py:172: error: Need type annotation for "sr1"  [var-annotated]
test.py:174: error: Dict entry 0 has incompatible type "Timestamp": "str"; expected "str": "ndarray[Any, Any]"  [dict-item]
test.py:178: error: Need type annotation for "sr2"  [var-annotated]
test.py:180: error: Dict entry 0 has incompatible type "Tuple[str, str]": "str"; expected "str": "ndarray[Any, Any]"  [dict-item]
Dr-Irv commented 10 months ago

Additional migration issues for the pd.Series constructor involve passing a dictionary. It fails with a Timestamp or multi-index key whereas before it did not:

I created a separate issue for this. #831

Dr-Irv commented 10 months ago

I am able to resolve 2.1.1 with Python 3.8 and while I plan to upgrade soon, it wouldn't help unless this gets fixed in the stubs library first. This issue affects pd.Series and not just pd.Index.

I just tried this with the latest release of the stubs (2.1.4.231218) and mypy 1.7.1, with python 3.9, and the Series lines pass fine, but there is a problem with Index that we need to fix up.

We won't update the stubs to support older versions of python, mypy, pyright, or pandas.

amgcc commented 10 months ago

We won't update the stubs to support older versions of python, mypy, pyright, or pandas.

Thats understandable. I appreciate you taking the time to fix this issue.