search

apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
https://kyuubi.apache.org/
Apache License 2.0
2.09k stars 913 forks source link

[FEATURE] Supports materialized views #1222

Open melin opened 3 years ago

melin commented 3 years ago

Code of Conduct

Search before asking

Describe the feature

Supports materialized views

Motivation

No response

Describe the solution

No response

Additional context

No response

Are you willing to submit PR?

ulysses-you commented 3 years ago

thank you @melin for the idea, look forward to your PR ! BTW can you provide some more details about this materialized views feature ?

melin commented 3 years ago

odps materialized view documentation： https://help.aliyun.com/document_detail/294513.html#section-wbz-zel-w69 community has this discussion and practice：https://www.iteblog.com/archives/9808.html

materialized view has many application scenarios，If the implementation is very valuable

ulysses-you commented 3 years ago

thank you @melin , that's a great work ! Please ping me if you get some trouble in Kyuubi community.

zinking commented 3 years ago

odps materialized view documentation： https://help.aliyun.com/document_detail/294513.html#section-wbz-zel-w69 community has this discussion and practice：https://www.iteblog.com/archives/9808.html

materialized view has many application scenarios，If the implementation is very valuable

isn't this related to the engine behind kyuubi, I'm wondering why this feature is requested from kyuubi though

pan3793 commented 3 years ago

Kyuubi also provides a moudle named Kyuubi Spark Extenstion, which extends catalyst and enrich the functionalities of Spark, technically speaking, we can cover this feature in such moudle.

zinking commented 3 years ago

Kyuubi also provides a moudle named Kyuubi Spark Extenstion, which extends catalyst and enrich the functionalities of Spark, technically speaking, we can cover this feature in such moudle.

that is still utterly weird, if the feature is kyuubi related I guess it is reasonable. but Materialized view ? why not direc to spark core?

pan3793 commented 3 years ago

The Kyuubi Community doesn't forbid any contribution to any other project like Spark, if the Materialized View feature gets into Spark, we will be happy to see that. In meantime, Kyuubi has such a module that can extend Spark catalyst, we are also glad to see people enrich it.