ibis-project / ibis

the portable Python dataframe library
https://ibis-project.org
Apache License 2.0
4.36k stars 540 forks source link

feat(pyspark): builtin udf support #9190

Closed ted0928 closed 2 weeks ago

ted0928 commented 2 weeks ago

Description of changes

support builtin udf by using @ibis.udf.scalar.builtin annotation in spark backend.

modified spark backend _register_udfs to accept builtin udf. modified spark compiler __sql_name__ to remain the origin function name.