MilanStaffehl / numdantic

Typing support for numpy arrays and numpy array validation with pydantic.
MIT License
1 stars 0 forks source link

Add a description to "Limitations" about dtypes that are not subclasses of `np.generic` #6

Closed MilanStaffehl closed 2 months ago

MilanStaffehl commented 2 months ago

Sometimes, type checkers will complain about certain dypes, since they are not actually typed as subtypes of np.generic in numpy. This is highly annoying, but inevitable, and so far there is little to be done about this. Example:

import numpy as np

def stringify_dtype(dtype: np.generic) -> str:
    return dtype.__name__

stringify_dtype(np.float64)  # <--- causes error on x64 architectures

The above example, run against mypy on a 64-bit architecture, results in the following error:

Argument 1 to "stringify_dtype" has incompatible type "type[floating[Any]]"; expected "generic"  [arg-type]

Might however be solved together with the problem regarding built-ins in the same place - it is effectively the same problem: numpy does not type classes as subtypes of np.generic that are treated as such on runtime. For built-ins, this is a mild annoyance - but for numpy scalars, this is actually damn near deal-breaking...!

I'll keep this a docs issue for now, investigating a fix is worth its own issue. Potentially one could get around the problem by writing a custom type that includes type[floating[Any]] etc. and then just ignore the fact that np.ndarray is typed in a way that will not allow this type (since it only accepts np.generic as dtype - which these apparently aren't).

MilanStaffehl commented 2 months ago

Actually, I will open an issue to fix this - if for nothing else, then to at least refer to it in the code after the # type: ignore instructions: #7

MilanStaffehl commented 2 months ago

I have to be honest: I am not sure what I was thinking when opening this issue? This does not seem to be a problem with the NDArray type, and more of a general numpy issue. After some checks, this issue does not occur for arrays typed as np.float64 and target type np.generic:

from numdantic import NDArray, Shape
import numpy as np

x: NDArray[Shape[int], np.float64] = np.array([1], dtype=np.float64)
y: NDArray[Shape[int], np.generic] = x

def some_func(x: NDArray[Shape[int], np.generic]) -> None:
    pass

some_func(x)

The above passes strict type checking just fine.

So, the lesson here is, I guess: "Don't write GitHub issues when you're tired"?