search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.22k stars 437 forks source link

[GLUTEN-8020] Remove the libhdfs3 installation script required for static linking #8013

Open JkSelf opened 23 hours ago

JkSelf commented 23 hours ago

What changes were proposed in this pull request?

We will only retain the dynamic libraries libhdfs.so or libhdfs3.so at runtime based on this benchmark here. So there is no need to keep the libhdfs3 installation script required for static linking.

How was this patch tested?

Existing compile scripts.

github-actions[bot] commented 23 hours ago

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

PHILO-HE commented 21 hours ago

@JkSelf, can we also remove ENABLE_HDFS option from some build scripts?

zhouyuan commented 19 hours ago

@JkSelf could you please help to link with the right Issue?

github-actions[bot] commented 4 hours ago

https://github.com/apache/incubator-gluten/issues/8020

PHILO-HE commented 3 hours ago

@JkSelf, can we also remove ENABLE_HDFS option from some build scripts?

Seems we still need it for deciding whether to register hdfs filesystem.

JkSelf commented 3 hours ago

@JkSelf, can we also remove ENABLE_HDFS option from some build scripts?

Seems we still need it for deciding whether to register hdfs filesystem.

Yes. We also need to pass the enable_hdfs to velox compile scripts.