search

substrait-io / substrait

A cross platform way to express data transformation, relational algebra, standardized record expression and plans.
https://substrait.io
Apache License 2.0
1.2k stars 157 forks source link

feat(proto): Add protobuf support for user-defined-function extension #233

Open rtpsw opened 2 years ago

rtpsw commented 2 years ago

Currently, Ibis has vectorized UDF nodes that are not handled by Ibis-Substrait (see https://github.com/ibis-project/ibis-substrait/issues/236). This issue is specific for the protobuf changes needed.

westonpace commented 1 year ago

This issue is somewhat vague (is this for embedding UDF implementation into the plan? For calling pre-registered UDFs in the consumer? etc?) Could you clarify? I know we've all done some work in this area over the last six months. Alternatively, we could close this and reopen issues / discussion at a later time.

rtpsw commented 1 year ago

This is for embedding vector UDF implementations - using the func member. See test_udf.py in Arrow for the scalar (rather than vector) UDF is currently implemented in a Substrait consumer.

AFAIK, this issue can be held for the time being. @icexelloss could bear on prioritization.