numpy ufuncs

[ev-br]

Stephan Hoyer on the ML:

You could actually use a mix of array_prepare and array_wrap to make (non-generalized) ufuncs work, e.g., for functions like np.sin:

• In array_prepare, return the non-fill values of the array concatenated with the fill value.
• In array_wrap, reshape all but the last element to build a new sparse array, using the last element for the new fill value.

This would be a neat trick and get you most of what you could hope for from numpy_ufunc.

Looks extremely cool, needs copious testing.

[ev-br]

Sadly, this does not seem to work. A stub at https://github.com/ev-br/sparr/commits/array_wrap fails with

In [1]: import numpy as np

In [2]: from sparr import MapArray as M

In [3]: m = M()

In [4]: np.sin(m)
('Array_prepare!', array(None, dtype=object), (<ufunc 'sin'>, (<0x0 sparse array of type '<class 'numpy.float64'>' with 0 stored elements.>,), 0))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-add0518d1dc2> in <module>()
----> 1 np.sin(m)

TypeError: __array_prepare__ must return an ndarray or subclass thereof which is otherwise identical to its input

The error seems to originate from https://github.com/numpy/numpy/blame/2af06c804931aae4b30bb3349bc60271b0b65381/numpy/core/src/umath/ufunc_object.c#L1362 which leaves little chance for this to ever work at all.

[ev-br]

On a second look though, there might be a chance for this with __array__ instead of __array_prepare.

[ev-br]

No, in fact, __array__ won't do:

• np.add(m, a) calls m.__array__. The latter then must return m.todense(), otherwise numpy detects a shape mismatch.
• np.asarray(m) calls m.__array__ but does not call m.__array_wrap__, so that there's no chance to reshape the result of __array__ back.

[shoyer]

sigh

It was a worth a try -- sorry I led you astray! I guess we'll have to wait for __numpy_ufunc__ then :(.

It looks like the output array for the ufunc is allocated (but not filled) before __array_prepare__ is called.