search

oap-project / gazelle_plugin

Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.
Apache License 2.0
256 stars 77 forks source link

coredump appears in udf development using third-party class libraries #1140

Closed ziyangRen closed 1 year ago

ziyangRen commented 1 year ago

Describe the bug We noticed the previously updated UDF development #945 ,In our actual development, we found that when the third-party library is not used, it can work normally. Once the third-party library is introduced, such as "libuuid.so" coredump will appear. We want to know whether a third-party class library can be used when developing new expressions, and if so, how we should use it.

PHILO-HE commented 1 year ago

For implementing new function with third-party lib, a so-called function holder should be used in arrow. The below PR link should be a good reference to you. https://github.com/oap-project/arrow/pull/97

zhouyuan commented 1 year ago

@ziyangRen

Is it convenient for you to share the call stack on the coredump? Usually there will be some clues in the call stack. Please also check if all the dependencies are installed on worker nodes.

thanks, -yuan

ziyangRen commented 1 year ago

thanks very much.We have now successfully implemented udf functions.