PowerGridModel / power-grid-model-io

Conversion tool for various grid data formats to power-grid-model
Mozilla Public License 2.0
14 stars 8 forks source link

Running mypy manually highlights multiple issues, while pre-commit reports none #72

Closed bramstoeller closed 1 year ago

pkuppens commented 1 year ago

Management summary

I think that this can be solved with a few steps:

Detail

I tried to understand what the problem is here.

It would be nice if the problem description would include the steps to reproduce the problem and show the expected results.

I don't know if I should do extra steps to verify that the pre-commit hook that runs/should run mypy works as expected.

At least I could try to run mypy manually myself:

(venv) D:\Alliander\code\power-grid-model-io>python -m mypy src
src\power_grid_model_io\converters\pandapower_converter.py:1283: error: Function "numpy.core.multiarray.arange" is not valid as a type  [valid-type]
src\power_grid_model_io\converters\pandapower_converter.py:1283: note: Perhaps you need "Callable[...]" or a callback protocol?
src\power_grid_model_io\converters\pandapower_converter.py:1304: error: Function "numpy.core.multiarray.array" is not valid as a type  [valid-type]
src\power_grid_model_io\converters\pandapower_converter.py:1304: note: Perhaps you need "Callable[...]" or a callback protocol?
src\power_grid_model_io\converters\pandapower_converter.py:1586:22: error: No overload variant of "__call__" of "_UFunc_Nin2_Nout1" matches argument types "Any", "None"  [call-overload]
src\power_grid_model_io\converters\pandapower_converter.py:1586:22: note: Possible overload variants:
src\power_grid_model_io\converters\pandapower_converter.py:1586:22: note:     def __call__(self, Union[int, float, complex, str, bytes, generic], Union[int, float, complex, str, bytes, generic], /, out: None = ..., *, where: Union[None, _SupportsArray[dtype[bool_]], _NestedSequence[_SupportsArray[dtype[bool_]]], bool, _NestedSequence[bool]] = ..., casting: Literal['no', 'equiv', 'safe', 'same_kind', 'unsafe'] = ..., order: Optional[Literal['K', 'A', 'C', 'F']] = ..., dtype: Union[dtype[Any], None, Type[Any], _SupportsDType[dtype[Any]], str, Union[Tuple[Any, int], Tuple[Any, Union[SupportsIndex, Sequence[SupportsIndex]]], List[Any], _DTypeDict, Tuple[Any, Any]]] = ..., subok: bool = ..., signature: Union[str, Tuple[Optional[str], Optional[str], Optional[str]]] = ..., extobj: List[Any] = ...) -> Any
src\power_grid_model_io\converters\pandapower_converter.py:1586:22: note:     def __call__(self, Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]], Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]], /, out: Union[None, ndarray[Any, dtype[Any]], Tuple[ndarray[Any, dtype[Any]]]] = ..., *, where: Union[None, _SupportsArray[dtype[bool_]], _NestedSequence[_SupportsArray[dtype[bool_]]], bool, _NestedSequence[bool]] = ..., casting: Literal['no', 'equiv', 'safe', 'same_kind', 'unsafe'] = ..., order: Optional[Literal['K', 'A', 'C', 'F']] = ..., dtype: Union[dtype[Any], None, Type[Any], _SupportsDType[dtype[Any]], str, Union[Tuple[Any, int], Tuple[Any, Union[SupportsIndex, Sequence[SupportsIndex]]], List[Any], _DTypeDict, Tuple[Any, Any]]] = ..., subok: bool = ..., signature: Union[str, Tuple[Optional[str], Optional[str], Optional[str]]] = ..., extobj: List[Any] = ...) -> ndarray[Any, dtype[Any]]
Found 3 errors in 1 file (checked 32 source files)

It indeed highlights multiple issues.

I think that the issues in lines 1283 and 1304 are small errors in the type hinting. arange is a function and not a data type. The function arange returns a ndarray, so my first guess would be to replace arange with ndarray (the errors go away, but I don't know if this is the desired solution).

Unfortunately, the last error, in line 1586 is more complex. It seems mypy has an issue with np.equal(attr_data, None): mypy inferred that attr_data is of type Any, and np.equaldoes not have an overload that checks ifAnyisNoneaccording tomypy` (it works just fine in practice).

So while np.equal(attr_data, None) is a valid check for a possible None, mypy does not seem to like it. There are a few solutions I can think of, but all have drawbacks:

  1. rewrite this function: we should not rewrite good code just to please tooling
  2. wrap the function: this may only relocate the problem, mypy may complain about the wrapper
  3. ignore this in mypy: starting to ignore warnings
  4. fix this in numpy: a lot of work for this single statement
  5. python -m mypy src --follow-imports skip does not show this error, maybe use skip in the tool.mypy configuration?