p2p-ld / numpydantic

Type annotations for specifying, validating, and serializing arrays with arbitrary backends in Pydantic (and beyond)
https://numpydantic.readthedocs.io/
MIT License
66 stars 1 forks source link

Union of numpydantic types #25

Closed uellue closed 1 month ago

uellue commented 1 month ago

First, thank you for implementing union types so quickly!

When trying it out, I came across this:

class TT(BaseModel):
    t: numpydantic.NDArray[numpydantic.Shape['2, 2'], numpydantic.dtype.Float32 | numpydantic.dtype.Float64]

Error:

---------------------------------------------------------------------------
InvalidArgumentsError                     Traceback (most recent call last)
Cell In[13], line 1
----> 1 class TT(BaseModel):
      2     t: numpydantic.NDArray[numpydantic.Shape['2, 2'], numpydantic.dtype.Float32 | numpydantic.dtype.Float64]

Cell In[13], line 2, in TT()
      1 class TT(BaseModel):
----> 2     t: numpydantic.NDArray[numpydantic.Shape['2, 2'], numpydantic.dtype.Float32 | numpydantic.dtype.Float64]

File [~/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/base_meta_classes.py:150](http://localhost:8888/home/weber/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/base_meta_classes.py#line=149), in SubscriptableMeta.__getitem__(cls, item)
    147 if getattr(cls, "_parameterized", False):
    148     raise NPTypingError(f"Type nptyping.{cls} is already parameterized.")
--> 150 args = cls._get_item(item)
    151 additional_values = cls._get_additional_values(item)
    152 assert hasattr(cls, "__args__"), "A SubscriptableMeta must have __args__."

File [~/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/ndarray.py:76](http://localhost:8888/home/weber/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/ndarray.py#line=75), in NDArrayMeta._get_item(cls, item)
     74 def _get_item(cls, item: Any) -> Tuple[Any, ...]:
     75     cls._check_item(item)
---> 76     shape, dtype = cls._get_from_tuple(item)
     77     return shape, dtype

File [~/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/ndarray.py:118](http://localhost:8888/home/weber/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/ndarray.py#line=117), in NDArrayMeta._get_from_tuple(cls, item)
    115 def _get_from_tuple(cls, item: Tuple[Any, ...]) -> Tuple[Shape, DType]:
    116     # Return the Shape Expression and DType from a tuple.
    117     shape = cls._get_shape(item[0])
--> 118     dtype = cls._get_dtype(item[1])
    119     return shape, dtype

File [~/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/ndarray.py:130](http://localhost:8888/home/weber/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/ndarray.py#line=129), in NDArrayMeta._get_dtype(cls, dtype_candidate)
    128 elif cls._is_literal_like(dtype_candidate):  # pragma: no cover
    129     structure_expression = dtype_candidate.__args__[0]
--> 130     dtype = Structure[structure_expression]
    131     check_type_names(dtype, dtype_per_name)
    132 elif isinstance(dtype_candidate, tuple):  # pragma: no cover

File [~/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/base_meta_classes.py:150](http://localhost:8888/home/weber/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/base_meta_classes.py#line=149), in SubscriptableMeta.__getitem__(cls, item)
    147 if getattr(cls, "_parameterized", False):
    148     raise NPTypingError(f"Type nptyping.{cls} is already parameterized.")
--> 150 args = cls._get_item(item)
    151 additional_values = cls._get_additional_values(item)
    152 assert hasattr(cls, "__args__"), "A SubscriptableMeta must have __args__."

File [~/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/base_meta_classes.py:216](http://localhost:8888/home/weber/miniconda3/envs/ltschema312/lib/python3.12/site-packages/numpydantic/vendor/nptyping/base_meta_classes.py#line=215), in ContainerMeta._get_item(cls, item)
    214 def _get_item(cls, item: Any) -> Tuple[Any, ...]:
    215     if not isinstance(item, str):
--> 216         raise InvalidArgumentsError(
    217             f"Unexpected argument of type {type(item)}, expecting a string."
    218         )
    220     if item in cls._known_expressions:
    221         # No need to do costly validations and normalizations if it has been done
    222         # before.
    223         return (item,)

InvalidArgumentsError: Unexpected argument of type <class 'type'>, expecting a string.

Can you reproduce it? Are the types supposed to be used like this?

sneakers-the-rat commented 1 month ago

That should work, executes fine here and there are a set of tests just like that here: https://github.com/p2p-ld/numpydantic/blob/1cf69eb18c57aa5ed425a0d925378f4994f10ef5/tests/conftest.py#L199-L204

can you print me a python -m pip freeze from that environment?

sneakers-the-rat commented 1 month ago

for the record the goal is to minimize special types, i'll add to the docs that when in doubt just use numpy types as the standard way to refer to things like this. there's a bit of subtlety to come re: mapping dtypes between array frameworks, but when i doubt i would treat numpy types as the lingua franca

uellue commented 1 month ago

Thank you for reproducing! I tried a fresh environment and it worked. For some reason I got numpydantic==1.6.0 in the old one.