Closed chipkent closed 11 months ago
Docs for guvectorize are bad. Here are some better ones. https://numba.pydata.org/numba-doc/latest/user/vectorize.html#vectorize https://numba.pydata.org/numba-doc/0.17.0/user/vectorize.html https://numba-users.continuum.narkive.com/7OAX8Suv/numba-guvectorize-with-fixed-size-output-array https://numpy.org/doc/stable/reference/c-api/generalized-ufuncs.html#generalized-universal-function-api
Numba's
guvectorize
annotation is extremely powerful. It provides a few key features not found invectorize
, which is currently supported by DH.The following code illustrates what is possible. The one notable unsupported feature from the numba side is fixed size outputs (https://github.com/numba/numba/issues/1668).
On the DH side, the query language needs to support outputs from the guvectorize functions. Currently, all guvectorize output columns are
java.lang.Object
, instead of being the proper scalar or array type.Making this fix will allow high performance creation of native java arrays that can be seamlessly be used in operations such as grouping. Also, the array support makes it easier to directly use inputs as numpy arrays.