search

seberg / experimental_user_dtypes

This repository houses experimental numpy DTypes using the new API (and related features). EXPECT THIS TO VANISH, I plan to move this to the NumPy org when the dust settles.
4 stars 2 forks source link

Create example for an abstract DType customizing array coercion #3

Open seberg opened 3 years ago

seberg commented 3 years ago

@mhvk to move the "off topic" comment here. Do you have an example of what you might like to see? Otherwise, this is just for me to try to think of an example.

(Could be a dicts as structured void, although structured voids are terrible at promoting right now...).

mhvk commented 3 years ago

Yes, I've been writing a StructuredUnit, which has a different unit for each element of a structure. If I want to associate this with an input given as a list of tuples to make a structured quantity, I know the structure, but not necessarily the type of each element (float, complex, float32 subarray). Right now, this is handled by explicitly passing the pieces through np.array - but it would be nice to pass the whole list in. See https://github.com/mhvk/astropy/blob/b7235978c3534eb336d50de8e659ca5e53f516de/astropy/units/structured.py#L198-L230

Aside: for the regular case, this would become nearly trivial if the unit is part of the dtype -- I'm quite looking forward to trying that! (An exception, I guess, would be if a user wants to use the same unit with, say, float32 or complex values. But really it would seem OK if that were to need some special methods/functions)

seberg commented 3 years ago

So, I think this is possible, but only if you create an (abstract) DType that already fully defines the structure. I.e. if you would have something like:

class CartesianValueDType(np.StructuredDType):
    x : np.Float64  # always float
    y : np.Float64
    value : np.dtype  # any dtype

With pre-defined fields, then it would already be possible. As an abstract DType, it would end up returning structured void dtype instances, though. And you have to create a DType class. (I am still playing with the idea of allowing abstract dtype instances as well as classes. I thought I can avoid it, but it does seem to open up some interesting possibilities, including here.)

I think I will try out the above in Cython here some time, when I need a break from string promotion ;).

The long-term goal should be to make the above work like a dataclass DType (i.e. allow creating a DType for a specific dataclass by typing it slightly stricter, maybe).

mhvk commented 3 years ago

That sounds nice: In my example, I should be able to quickly define a dtype like the one you suggest.