search

pandas-dev / pandas

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
https://pandas.pydata.org
BSD 3-Clause "New" or "Revised" License
43.65k stars 17.92k forks source link

TYPING: Unreachable code #27396

Open simonjayhawkins opened 5 years ago

simonjayhawkins commented 5 years ago

mypy 0.720 introduced --warn-unreachable http://mypy-lang.blogspot.com/2019/07/mypy-0720-released.html 

$ mypy pandas --warn-unreachable
pandas\core\computation\expressions.py:68: error: Statement is unreachable
pandas\core\computation\expressions.py:123: error: Statement is unreachable
pandas\core\dtypes\dtypes.py:98: error: Statement is unreachable
pandas\core\dtypes\dtypes.py:102: error: Statement is unreachable
pandas\core\dtypes\dtypes.py:316: error: Statement is unreachable
pandas\core\dtypes\dtypes.py:541: error: Right operand of 'and' is never evaluated
pandas\core\dtypes\dtypes.py:543: error: Statement is unreachable
pandas\core\indexers.py:105: error: Statement is unreachable
pandas\core\indexers.py:233: error: Statement is unreachable
pandas\core\arrays\array_.py:230: error: Statement is unreachable
pandas\core\arrays\_ranges.py:40: error: Statement is unreachable
pandas\core\arrays\period.py:243: error: Statement is unreachable
pandas\core\indexes\interval.py:1212: error: Statement is unreachable
pandas\core\computation\pytables.py:93: error: Left operand of 'and' is always true
pandas\core\computation\pytables.py:103: error: Left operand of 'and' is always true
pandas\core\indexing.py:185: error: Left operand of 'or' is always false
pandas\core\indexing.py:246: error: Statement is unreachable
pandas\core\internals\managers.py:1491: error: Statement is unreachable
pandas\core\internals\managers.py:1505: error: Statement is unreachable
pandas\core\internals\managers.py:1517: error: Statement is unreachable
pandas\io\json\_normalize.py:255: error: Statement is unreachable
pandas\io\json\_normalize.py:284: error: Statement is unreachable
(pandas-dev)

should we include this in ci\code_checks.sh?

WillAyd commented 5 years ago

I think a good idea in the long run. Are these ones now actually unreachable?

simonjayhawkins commented 5 years ago

Are these ones now actually unreachable?

i've not looked in detail, but may be that the type annotation is too restrictive for those public methods.

jbrockmendel commented 5 years ago

The computation.expressions ones are false-positives, at least one of the dtypes.dtypes is false-positive. Pretty sure the same for core.indexers

simonjayhawkins commented 5 years ago

The computation.expressions ones are false-positives

Could the code or type annotations be changed to suppress the warning?

If not, it would make it difficult to add --warn-unreachable to ci\code_checks.sh

WillAyd commented 5 years ago

The computation.expressions ones are false-positives

Not super familiar with that module but where is the value of _TEST_MODE being set? AFAICT it's only called in set_test_mode in that same module which isn't public and not called from anywhere else except outside of a test. So I think it is actually dead code?

jbrockmendel commented 5 years ago

AFAICT it's only called in set_test_mode in that same module which isn't public and not called from anywhere else except outside of a test

It is declared global on L247, so when it is set on L248 that sets it in the module-scope too.

WillAyd commented 5 years ago

Yea those lines are in the set_test_mode function I mentioned that doesn't get called except from the test suite though

I think there's just a lot of test code crept in there. Can take a look later and push a PR if it makes sense

fangchenli commented 4 years ago

mypy=0.790

mypy pandas --warn-unreachable
pandas/core/arrays/_ranges.py:43: error: Statement is unreachable  [unreachable]
pandas/core/computation/parsing.py:41: error: unused 'type: ignore' comment
pandas/core/dtypes/base.py:250: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/base.py:426: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/base.py:430: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/dtypes.py:656: error: Left operand of 'and' is always true  [unreachable]
pandas/core/dtypes/dtypes.py:875: error: Left operand of 'and' is always true  [unreachable]
pandas/core/dtypes/dtypes.py:888: error: Statement is unreachable  [unreachable]
pandas/core/computation/expressions.py:67: error: Statement is unreachable  [unreachable]
pandas/core/computation/expressions.py:117: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/common.py:949: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/common.py:1227: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/common.py:1685: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/common.py:1792: error: Statement is unreachable  [unreachable]
pandas/io/common.py:217: error: Right operand of 'and' is never evaluated  [unreachable]
pandas/io/common.py:231: error: Right operand of 'and' is never evaluated  [unreachable]
pandas/io/common.py:244: error: Left operand of 'or' is always false  [unreachable]
pandas/io/common.py:452: error: Unsupported operand types for + ("List[Optional[str]]" and "List[str]")  [operator]
pandas/core/indexers.py:165: error: Statement is unreachable  [unreachable]
pandas/core/indexers.py:305: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/cast.py:237: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/cast.py:752: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/cast.py:1789: error: Statement is unreachable  [unreachable]
pandas/core/dtypes/cast.py:1795: error: Statement is unreachable  [unreachable]
pandas/core/missing.py:54: error: Statement is unreachable  [unreachable]
pandas/core/common.py:90: error: Statement is unreachable  [unreachable]
pandas/core/common.py:257: error: Statement is unreachable  [unreachable]
pandas/core/common.py:490: error: Statement is unreachable  [unreachable]
pandas/plotting/_matplotlib/style.py:43: error: Subclass of "List[Any]" and "str" cannot exist: would have incompatible method signatures  [unreachable]
pandas/plotting/_matplotlib/style.py:44: error: Statement is unreachable  [unreachable]
pandas/plotting/_matplotlib/style.py:59: error: Subclass of "List[Any]" and "str" cannot exist: would have incompatible method signatures  [unreachable]
pandas/plotting/_matplotlib/style.py:59: error: Right operand of 'and' is never evaluated  [unreachable]
pandas/plotting/_matplotlib/style.py:61: error: Statement is unreachable  [unreachable]
pandas/core/apply.py:108: error: Statement is unreachable  [unreachable]
pandas/core/sorting.py:73: error: Statement is unreachable  [unreachable]
pandas/core/ops/__init__.py:232: error: Statement is unreachable  [unreachable]
pandas/core/arrays/numpy_.py:84: error: Statement is unreachable  [unreachable]
pandas/core/arrays/numpy_.py:218: error: Statement is unreachable  [unreachable]
pandas/core/arrays/period.py:766: error: Statement is unreachable  [unreachable]
pandas/core/arrays/period.py:810: error: Statement is unreachable  [unreachable]
pandas/core/arrays/period.py:1077: error: Right operand of 'and' is never evaluated  [unreachable]
pandas/core/arrays/period.py:1084: error: If condition is always true  [unreachable]
pandas/core/arrays/datetimes.py:2250: error: Left operand of 'and' is always true  [unreachable]
pandas/core/indexes/base.py:388: error: Statement is unreachable  [unreachable]
pandas/core/indexes/base.py:898: error: Statement is unreachable  [unreachable]
pandas/core/arrays/timedeltas.py:519: error: Statement is unreachable  [unreachable]
pandas/core/arrays/timedeltas.py:528: error: Statement is unreachable  [unreachable]
pandas/core/arrays/timedeltas.py:576: error: Statement is unreachable  [unreachable]
pandas/core/arrays/timedeltas.py:581: error: Statement is unreachable  [unreachable]
pandas/core/arrays/timedeltas.py:622: error: Statement is unreachable  [unreachable]
pandas/core/arrays/timedeltas.py:940: error: Statement is unreachable  [unreachable]
pandas/core/arrays/interval.py:554: error: Statement is unreachable  [unreachable]
pandas/core/arrays/interval.py:1104: error: Statement is unreachable  [unreachable]
pandas/core/arrays/interval.py:1109: error: Statement is unreachable  [unreachable]
pandas/core/arrays/integer.py:383: error: Statement is unreachable  [unreachable]
pandas/core/arrays/integer.py:562: error: Statement is unreachable  [unreachable]
pandas/core/arrays/floating.py:301: error: Statement is unreachable  [unreachable]
pandas/core/arrays/categorical.py:1676: error: Statement is unreachable  [unreachable]
pandas/core/arrays/boolean.py:310: error: Statement is unreachable  [unreachable]
pandas/core/arrays/sparse/array.py:1324: error: Statement is unreachable  [unreachable]
pandas/core/internals/blocks.py:1696: error: Left operand of 'and' is always true  [unreachable]
pandas/core/internals/blocks.py:2025: error: Left operand of 'and' is always true  [unreachable]
pandas/core/internals/blocks.py:2074: error: Left operand of 'and' is always true  [unreachable]
pandas/core/internals/blocks.py:2377: error: Statement is unreachable  [unreachable]
pandas/core/indexes/numeric.py:52: error: Statement is unreachable  [unreachable]
pandas/core/indexes/range.py:722: error: Left operand of 'and' is always true  [unreachable]
pandas/core/indexes/range.py:724: error: Statement is unreachable  [unreachable]
pandas/core/indexes/range.py:733: error: Right operand of 'and' is never evaluated  [unreachable]
pandas/core/indexes/range.py:830: error: Statement is unreachable  [unreachable]
pandas/core/indexes/range.py:834: error: Statement is unreachable  [unreachable]
pandas/core/indexes/multi.py:530: error: Statement is unreachable  [unreachable]
pandas/core/indexes/multi.py:2894: error: If condition is always true  [unreachable]
pandas/core/indexes/category.py:205: error: Left operand of 'and' is always true  [unreachable]
pandas/core/indexes/category.py:287: error: Statement is unreachable  [unreachable]
pandas/core/tools/datetimes.py:547: error: Statement is unreachable  [unreachable]
pandas/core/indexes/period.py:293: error: Statement is unreachable  [unreachable]
pandas/core/indexes/interval.py:1023: error: Statement is unreachable  [unreachable]
pandas/core/indexing.py:1830: error: Statement is unreachable  [unreachable]
pandas/core/indexing.py:2006: error: Statement is unreachable  [unreachable]
pandas/core/indexing.py:2015: error: Statement is unreachable  [unreachable]
pandas/core/indexing.py:2278: error: Statement is unreachable  [unreachable]
pandas/core/aggregation.py:483: error: Value of type variable "_LT" of "sorted" cannot be "Optional[Hashable]"  [type-var]
pandas/core/aggregation.py:606: error: Value of type variable "_LT" of "sorted" cannot be "Optional[Hashable]"  [type-var]
pandas/core/aggregation.py:634: error: Subclass of "Dict[Optional[Hashable], Any]" and "List[Any]" cannot exist: would have incompatible method signatures  [unreachable]
pandas/core/aggregation.py:635: error: Statement is unreachable  [unreachable]
pandas/core/window/rolling.py:186: error: If condition is always true  [unreachable]
pandas/core/window/rolling.py:202: error: Left operand of 'and' is always true  [unreachable]
pandas/core/window/rolling.py:1234: error: Statement is unreachable  [unreachable]
pandas/core/internals/construction.py:293: error: Statement is unreachable  [unreachable]
pandas/core/internals/construction.py:540: error: Left operand of 'and' is always true  [unreachable]
pandas/core/internals/construction.py:581: error: Statement is unreachable  [unreachable]
pandas/core/internals/construction.py:639: error: Statement is unreachable  [unreachable]
pandas/core/reshape/concat.py:342: error: Statement is unreachable  [unreachable]
pandas/io/formats/format.py:423: error: Statement is unreachable  [unreachable]
pandas/io/formats/format.py:1724: error: Statement is unreachable  [unreachable]
pandas/core/generic.py:259: error: Statement is unreachable  [unreachable]
pandas/core/generic.py:531: error: If condition in comprehension is always true  [unreachable]
pandas/core/generic.py:531: error: Subclass of "str" and "int" cannot exist: would have incompatible method signatures  [unreachable]
pandas/core/generic.py:6283: error: Statement is unreachable  [unreachable]
pandas/core/generic.py:6312: error: Statement is unreachable  [unreachable]
pandas/core/generic.py:9761: error: Subclass of "str" and "timedelta" cannot exist: would have incompatible method signatures  [unreachable]
pandas/core/series.py:939: error: Statement is unreachable  [unreachable]
pandas/core/series.py:2604: error: Statement is unreachable  [unreachable]
pandas/core/frame.py:496: error: Statement is unreachable  [unreachable]
pandas/core/frame.py:704: error: Statement is unreachable  [unreachable]
pandas/core/frame.py:1793: error: Statement is unreachable  [unreachable]
pandas/core/frame.py:1998: error: Statement is unreachable  [unreachable]
pandas/core/frame.py:7334: error: Statement is unreachable  [unreachable]
pandas/core/frame.py:8704: error: Left operand of 'and' is always true  [unreachable]
pandas/io/parsers.py:3792: error: Statement is unreachable  [unreachable]
pandas/core/groupby/ops.py:270: error: Incompatible types in assignment (expression has type "Index", variable has type "Iterator[Any]")  [assignment]
pandas/core/groupby/ops.py:270: note: 'Index' is missing following 'Iterator' protocol member:
pandas/core/groupby/ops.py:270: note:     __next__
pandas/core/groupby/grouper.py:522: error: Statement is unreachable  [unreachable]
pandas/core/groupby/grouper.py:858: error: Statement is unreachable  [unreachable]
pandas/core/groupby/groupby.py:2782: error: Statement is unreachable  [unreachable]
pandas/io/html.py:1077: error: Subclass of "int" and "Integral" cannot exist: would have incompatible method signatures  [unreachable]
pandas/io/html.py:1077: error: Subclass of "slice" and "Integral" cannot exist: would have incompatible method signatures  [unreachable]
pandas/io/html.py:1077: error: Right operand of 'and' is never evaluated  [unreachable]
pandas/io/excel/_base.py:356: error: Statement is unreachable  [unreachable]
pandas/core/groupby/generic.py:1046: error: Left operand of 'and' is always true  [unreachable]
pandas/core/groupby/generic.py:1203: error: Statement is unreachable  [unreachable]
pandas/core/groupby/generic.py:1232: error: If condition is always true  [unreachable]
pandas/core/reshape/pivot.py:428: error: Right operand of 'or' is never evaluated  [unreachable]
pandas/core/reshape/pivot.py:432: error: Statement is unreachable  [unreachable]
pandas/io/stata.py:1398: error: Statement is unreachable  [unreachable]
pandas/io/stata.py:1447: error: Statement is unreachable  [unreachable]
pandas/io/stata.py:1588: error: Statement is unreachable  [unreachable]
pandas/io/pytables.py:552: error: Statement is unreachable  [unreachable]
pandas/io/pytables.py:1000: error: Statement is unreachable  [unreachable]
pandas/io/pytables.py:1377: error: Statement is unreachable  [unreachable]
pandas/io/pytables.py:1983: error: Left operand of 'and' is always true  [unreachable]
pandas/io/pytables.py:3737: error: Statement is unreachable  [unreachable]
pandas/io/pytables.py:3995: error: Subclass of "int" and "List[Any]" cannot exist: would have incompatible method signatures  [unreachable]
pandas/io/pytables.py:3996: error: Statement is unreachable  [unreachable]
pandas/io/pytables.py:4540: error: Subclass of "int" and "List[Any]" cannot exist: would have incompatible method signatures  [unreachable]
pandas/io/pytables.py:4541: error: Statement is unreachable  [unreachable]
pandas/io/pytables.py:5006: error: Left operand of 'or' is always false  [unreachable]
pandas/io/pytables.py:5006: error: Subclass of "Sequence[int]" and "str" cannot exist: would have inconsistent method resolution order  [unreachable]
pandas/io/formats/excel.py:172: error: Statement is unreachable  [unreachable]
pandas/io/formats/excel.py:173: error: Subclass of "str" and "Dict[Any, Any]" cannot exist: would have incompatible method signatures  [unreachable]
pandas/io/formats/excel.py:174: error: Statement is unreachable  [unreachable]
pandas/io/excel/_odswriter.py:187: error: Statement is unreachable  [unreachable]
pandas/core/reshape/merge.py:617: error: Subclass of "bool" and "str" cannot exist: would have incompatible method signatures  [unreachable]
pandas/core/reshape/merge.py:618: error: Statement is unreachable  [unreachable]
pandas/io/formats/style.py:169: error: Left operand of 'or' is always false  [unreachable]
pandas/io/formats/style.py:1192: error: Left operand of 'or' is always false  [unreachable]
pandas/io/formats/style.py:1656: error: Statement is unreachable  [unreachable]
pandas/io/formats/style.py:1664: error: Statement is unreachable  [unreachable]
pandas/io/formats/csvs.py:314: error: Subclass of "Hashable" and "List[Any]" cannot exist: would have incompatible method signatures  [unreachable]
pandas/io/formats/csvs.py:314: error: Right operand of 'and' is never evaluated  [unreachable]
pandas/io/formats/csvs.py:315: error: Statement is unreachable  [unreachable]
pandas/plotting/_matplotlib/core.py:191: error: Statement is unreachable  [unreachable]
pandas/plotting/_matplotlib/hist.py:418: error: Statement is unreachable  [unreachable]
pandas/tseries/holiday.py:430: error: Right operand of 'or' is never evaluated  [unreachable]
pandas/tests/test_flags.py:44: error: Statement is unreachable  [unreachable]
pandas/tests/series/test_internals.py:245: error: Statement is unreachable  [unreachable]
pandas/tests/io/test_gbq.py:44: error: Left operand of 'or' is always false  [unreachable]
pandas/tests/frame/test_constructors.py:1083: error: Statement is unreachable  [unreachable]
Found 157 errors in 65 files (checked 1057 source files)
jbrockmendel commented 2 years ago

Running mypy pandas --warn-unreachable the first three complaints i manually inspect are all false-positives. Do we have any precedent for this finding something accurately?

MarcoGorelli commented 2 years ago

Running mypy pandas --warn-unreachable the first three complaints i manually inspect are all false-positives. Do we have any precedent for this finding something accurately?

I don't know about mypy's flag, but I tried running vulture pandas --min-confidence 100 and found

https://github.com/pandas-dev/pandas/blob/9a4fcea8de798938a434fcaf67a0aa5a46b76b5b/pandas/tests/io/test_common.py#L557-L560

which seems legit