search

Alluxio / alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud
https://www.alluxio.io
Apache License 2.0
6.73k stars 2.92k forks source link

Compute Integration for Apache Impala #13484

Open LuckyNemo opened 3 years ago

LuckyNemo commented 3 years ago

Page Now alluxio has integrated Spark/MR/Hbase/Flink/Hive/Trino/Presto/TF and so on. But official document lack of support for Apache Impala, can u solve it rather than some other non-official material on the website!

Summary What is the defect and suggested improvement

LuckyNemo commented 3 years ago

@apc999

LuckyNemo commented 3 years ago

@zuston

zuston commented 3 years ago

As i know, Impala don't support Alluxio proto with HCFS interface. Maybe it should be supported by Impala.

Xinshiyou commented 3 years ago

good feature

linzhouzhi commented 3 years ago

very good feature

ColdZoo commented 3 years ago

Hi my folk alluxioers,

I am from Tencent Beacon, my teammate and I volunteer to populate this part of documentation.

Alluxio works as Impala's high performance reading cache in our clusters, saving up to 40% of Disk IOs of DataNodes. We are happy to share our findings.

@zuston So as to Impala's HDFS Scanner compatibility, there is an accepted patch which is uploaded by our team. This patch will be included in Impala 4.0. But interested readers can have it by cherry-picking from latest development branch now.

More information about our technical approach (Impala + Alluxio) is here

This is my first time contributing to alluxio doc, so I may need a bit time to investigate our guides and doc tool. I will upload a PR as soon as possible.

Thanks alluxio for making our life easier.

zuston commented 3 years ago

@ColdZoo Sounds great! Looking forward to your PR about Alluxio&Impala integration doc.

maobaolong commented 3 years ago

@ColdZoo Thanks for working on this, Look forward to your PR.

apc999 commented 3 years ago

@ColdZoo once you contributed your Doc back, we will send you a small gift package to appreciate your contribution! thanks in advance

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.