More documentation on the extent to which mypy enforces the types

[anieuwland]

Thanks for this impressive library. I feel it is really important for maintainable array-heavy codebases. I am running into the issue however that mypy does not seem to do anything with the type hints. I would expect it to complain if I gave a variable annotated with the type NDArray[Shape["2", "2"], UInt16] to a function with the signature func4(param1: NDArray[Shape["2", "3"], Float32]) -> ....

In the FAQ / documentation I could not find anywhere to what extent mypy actually enforces correct usage. Could you add a word on that?

Below I added a script that I would expect mypy to complain about. This is with Python 3.8, nptyping 2.2.0 and mypy 0.971.

from nptyping import NDArray, Shape, Float32, UInt16
import numpy as np

var1: NDArray[Shape["2", "2"], UInt16] = np.array([[1,2], [3,4]])

def func1(param1: NDArray[Shape["2", "2"], UInt16]) -> NDArray[Shape["2", "2"], UInt16]:
    return param1

def func2(param1: NDArray[Shape["2", "2"], UInt16]) -> NDArray[Shape["2", "3"], UInt16]:
    return param1

def func3(param1: NDArray[Shape["2", "3"], UInt16]) -> NDArray[Shape["2", "3"], UInt16]:
    return param1

def func4(param1: NDArray[Shape["2", "3"], Float32]) -> NDArray[Shape["2", "3"], Float32]:
    return param1

func1(var1)
func2(var1)
func3(var1)
func4(var1)

Output (I expected at least 3 issues):

❯ mypy test.py 
Success: no issues found in 1 source file

[ramonhagenaars]

Hi @anieuwland ! Happy to read that this library is of use to you!

I agree with you that it would be great for MyPy to do (all) the type checking. This is however not possible due to the dynamic nature of numpy, while MyPy is a static type checker.

Take this example:

import numpy as np

arr1 = np.array([1, 2, 3])  # dtype is now int32 or int64
arr2 = arr1 + .0  # dtype is now float64

This is beyond MyPy. As far as I can tell - and I hope to be proven wrong some day - the only thing that MyPy can check, is whether some variable or argument is an ndarray or not.

I just now wrote something in the FAQ of the newest release on this.

If you're ok with another 3rd party library and with adding some annotations, you could check out beartype. It will type check your functions for you, albeit on runtime, but with little overhead:

from beartype import beartype

from nptyping import NDArray, Shape, UInt16

@beartype
def func1(param1: NDArray[Shape["2, 2"], UInt16]) -> NDArray[Shape["2, 2"], UInt16]:
    return param1

func1("This ain't no array")

This will give:

<stacktrace>
beartype.roar.BeartypeCallHintParamViolation: @beartyped func1() parameter param1="This ain't no array" violates type hint NDArray[Shape['2, 2'], UShort], as "This ain't no array" not instance of <protocol "nptyping.base_meta_classes.NDArray">.

By the way, note that the syntax in your Shape is incorrect. It should not be Shape["2", "2"], but Shape["2, 2"]. 😉

[anieuwland]

By the way, note that the syntax in your Shape is incorrect. It should not be Shape["2", "2"], but Shape["2, 2"]. Oeps! Thanks :)

Also thanks for the beartype. Will check it out for the important non-hot functions! Just for my understanding, could you explain why the following doesn't work with mypy's static analysis?

In my code above I have the following snippet:

var1: NDArray[Shape["2, 2"], UInt16] = np.array([[1,2], [3,4]])
def func4(param1: NDArray[Shape["2, 3"], Float32]) -> NDArray[Shape["2, 3"], Float32]:
    return param1
func4(var1)

I have statically annotated var1 to be of a different type than the parameter of func4. Since variable / parameters types are statically set (and indeed thrown away at run-time, I thought), I would expect mypy to be able to check that they are the same. Why can't it?

I notice if I hover over the variable in vscode that it says the type is Any despite my annotation. Is that why?

[smheidrich]

@anieuwland Given that specific snippet you posted, Mypy does complain, at least for me:

$ mypy example.py 
example.py:7: error: Argument 1 to "func4" has incompatible type "ndarray[Any, dtype[unsignedinteger[_16Bit]]]"; expected "ndarray[Any, dtype[floating[_32Bit]]]"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

But I still agree with this issue because if I replace all the Float32s with UInt16s so the dtypes are all the same, it no longer complains, despite the shape mismatch...