QueryPlan Optimizer aims to run TPCH 100x faster

chdb-io / chdb

chDB is an in-process OLAP SQL Engine 🚀 powered by ClickHouse

https://clickhouse.com/docs/en/chdb

Apache License 2.0

2.03k stars 72 forks source link

QueryPlan Optimizer aims to run TPCH 100x faster #116

Open auxten opened 12 months ago

auxten commented 12 months ago

Still in a very early stage. This PR is only for tracking progress.

djouallah commented 12 months ago

wait are you going to build a new Query optimizer from scratch ?

auxten commented 12 months ago

wait are you going to build a new Query optimizer from scratch ?

No, transplant optimizer from ByConity which has forked from ClickHouse v21.8

alanpaulkwan commented 12 months ago

Is there any way to implement this in the main branch of Clickhouse as well? Would be huge

auxten commented 12 months ago

Is there any way to implement this in the main branch of Clickhouse as well? Would be huge

Perhaps later, even though it has already taken me months, I am unsure if this will be successful.😅

djouallah commented 12 months ago

@auxten you can do it !!!

alexey-milovidov commented 12 months ago

It makes sense to also send this PR to the main repository - interesting if it will pass the tests. Although I expect it is going to be difficult...

alexey-milovidov commented 11 months ago

One downside: if it is not tested by ClickHouse CI, it will most likely contain a ton of bugs, and having these bugs will lead to reputational risks for ClickHouse. It means - we have to send a PR to the main ClickHouse repository.

lmangani commented 11 months ago

One downside: if it is not tested by ClickHouse CI, it will most likely contain a ton of bugs, and having these bugs will lead to reputational risks for ClickHouse. It means - we have to send a PR to the main ClickHouse repository.

For sure. This will be considered highly experimental and it won't have mainstream ambitions unless we all agree the results are overwhelmingly positive. On the other hand as a fork, we can afford taking some early risks the mainline project cannot 😉

auxten commented 11 months ago

Absolutely agree with @alexey-milovidov. As a data tool the quality is the most important thing. This branch is very very experimental. I have never expected it would draw so much attention. I'm not even very confident on this could barely run. The quality of chdb depends mostly on ClickHouse. If this experiment could run(god bless), I will try to make a PR to clickhouse main repo. If I can manage it to pass all the ClickHouse tests and reviewed(god bless again). Then we can say it works. It may take years, but I am happy working on this.

CLAassistant commented 10 months ago

All committers have signed the CLA.