datafuselabs / databend

๐——๐—ฎ๐˜๐—ฎ, ๐—”๐—ป๐—ฎ๐—น๐˜†๐˜๐—ถ๐—ฐ๐˜€ & ๐—”๐—œ. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
https://docs.databend.com
Other
7.31k stars 704 forks source link

feat(query): add python udf script runtime #15494

Closed sundy-li closed 5 days ago

sundy-li commented 2 weeks ago

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  1. add python script runtime
  2. disable bendpy cause it's conflicted with python script runtime

Now we don't support python udf by default features.

We can enable it by using:

cargo build --bin databend-query --features "default,python-udf"

And the binary will contains the libpython3.12.so.1.0 link:

root@3d7690dd6269:/workspace# ldd ./target/debug/databend-query
        linux-vdso.so.1 (0x00007ffca2b96000)
        libpython3.12.so.1.0 => /usr/local/lib/libpython3.12.so.1.0 (0x0000787000993000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007870007c6000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007870007a9000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000078700078f000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x000078700076d000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x0000787000629000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x0000787000621000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000078700044d000)
        /lib64/ld-linux-x86-64.so.2 (0x000078701e532000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x0000787000448000)

Tests

Type of change


This change isโ€‚Reviewable

sundy-li commented 2 weeks ago

closing because we don't want to have /usr/lib/libpython3.12.so.1.0 dynamic link in binary

BohuTANG commented 5 days ago

Could we add some logic tests for the Python script to prevent it from failing unexpectedly? This would be crucial for users running the python script on the production. cc @sundy-li @everpcpc